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MULTIPLE RANK ORDER TYPAL ANALYSIS FOR THE 
ISOLATION OF INDEPENDENT TYPES 


LOUIS L. McQUITTY 
Michigan State University 


Wueruer or not types are found to exist depends in part on how 
they are defined. Quantitative methods for the isolation of types 
imply definitions of types. In the search for types, it is therefore 
helpful to have several quantitative methods, each related to a 
reasonable definition of types. The several methods could then be 
applied to determine which yields the most fruitful types in terms 
of some specified criteria. 

It would be still better if a single method were available which 
would both isolate and define all reasonable types reflected in a 

+ set of data. Pending availability of the latter alternative it is helpful 
.to proceed with development of methods under the former alterna- 
ae *tive, and this course of action might itself lead to a realization of 
J the latter alternative. 
We have already described and illustrated Multiple Rank Order 
Typal Analysis for the Isolation of Intersecting Types (McQuitty, 
1965). The present paper develops and illustrates an associated ver- 


sion of Rank Order Typal Analysis for the Isolation of Independ- 
ent Types. 


Theory in the Development of Methods 
Intersecting Types and Typologies 


In our development of the version for the isolation of inter- 
secting types, we applied several assumptions. We assumed that 
several, or even many, sets of pressures act on the developing 
person (institution or other object). Each set, or at least a few of 
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them, tends toward the development of types. Consequently, the 
individual is not a member of just one type but rather of many im- 
perfect (incomplete) types. However, these types cannot be seen 
with the “unaided eye”; the several types have become inter- 
mingled to such an extent that they are hidden. 

The version for isolating intersecting types serves to disentangle 
and display the several types. It is not, however, applied only to the 
single individual; it is applied to matrices of individuals and dis- 
plays multiple typologies, showing each person’s location in each 
typology and at least some of the significant characteristics which 
define him there. 


Independent Types and Typologies 


There is reason to believe that diverse pressures sometimes have 
more independent effects than are indicated above, at least in the 
case of some people. It is possible that each of several major pres- 
sures may have an effect on the developing personality and that each 
effect may be relatively independent of every other effect, thus 
producing relatively independent types and typologies. This 
kind of a result is implied in the personality descriptions of the 
mentally ill and to a lesser extent in the personalities of the nor- 


mal by those whose theories developed out of psychopathology, 


primarily. 
Simplicity of Structure 


Another argument in favor of the Independent Version is the 
fact that it yields a simpler model; each characteristic of a person 
is used only once in classifying him as compared with the possibil- 
ity of multiple usage in the case*of the Intersecting Versions. 


A Review of Rank Order Typal Analysis 


“Rank Order Typal Analysis defines a type as a category of 
people of such a nature that everyone in the category is more like 
everyone else in the category than he is like anyone in any other 
category... . 

“It is a simple matter to examine a matrix of interassociations 
between people in such a fashion as to isolate all of the categories 
which qualify as types. The task is accomplished by examining 
the matrix serially one column at a time. 
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“The first step is to take a matrix such as the one shown in 
Table 1 and convert it into a matrix of ranks as reported in Table 2 


TABLE 1 
Hypothetical Associations between People 


x y zZ 

w 71 60 65 35 

x 60 72 50 40 

y 65 50 73 45 

z 35 40 45 74 
TABLE 2 


CEEE] 


to show, for example, that the person most like v is x himself and 
then w, y, and z in that order. 

“Tf « forms a type with any one person of Table 2, it must be 
with the Person w, who is second most like him (x being most 
like himself). In order to examine this possibility, we form a sub- 
matrix of « with w, Table 3, using the ranks from Table 2. 
Person x does not form a type with Person w; the submatrix con- 
tains a rank larger than the number of persons in the matrix. Since 
Person x does not form a type of two persons with the one person 


TABLE 3 
A Submatriz from Table 2 


TABLE 4 
A Submatriz from Table 2 
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second most like x (x being most like himself), Person x does not 
form a type of two persons with any other one person of Table 2. 

“Tf æ forms a type with any two persons of Table 2, it must be 
with the persons second and third most like x. These are w 
and y. A submatrix of w, 2, and y is formed, Table 4. It con- 
tains no rank larger than 3. It proves, therefore, that x forms a type 
of three people with w and y and furthermore forms no other type 
of three people from those of Table 2. 

“The analysis proceeds in this fashion until Column z is com- 
pleted. In case of a 4 x 4 matrix, as in Table 2, a column has been 
completed when three variables have been included in a sub- 
matrix, as they have in Table 4; the analysis for æ was completed 
in the above operations. 

“Tp the general case, Column i of an nXn matrix has been com- 
pleted when a submatrix of n—1 variables has derived from a study 
of Column i. If all of the variables of a matrix were included in a 
submatrix, they would by necessity meet the above criterion but 
the criterion would no longer operate as a test of the presence of 
a type. 

“Qther columns of the matrix are analyzed in a similar fashion. 
In order to save time, steps should be taken to avoid duplications 
in the types isolated. For example, if Person x forms a type of 
three persons with Persons w and y, then each of these latter per- 
sons will yield the same type; they, therefore, need not be ex- 
amined for this possibility. 

“Tn case of ties of more than two indices, it is helpful to follow 
a method of assigning ranks which deviate from the usual method. 
In the new method, if Rank 7, for example, is followed by three 
tied indices such as 23, 23, 23, and then a larger index, say of 25, 
the three tied indices are assigned a rank of 10—the highest of the 
ranks in the series required for the three tied entries. The next 
entry is assigned a rank of 11. 

“Tp the general case, tied indices are each assigned the highest 
rank of the series required to cover the tied values. This approach 
prevents, for example, three tied cases from first forming three 
types of two cases each and then a type of three cases when all 
three cases are equally alike. 

“Once the data have been analyzed in the fashion outlined above, 
all possible types have been isolated, i.e., all possible types which 
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derive from all of the data, with every item equal in weight to 
every other item; other types can possibly be isolated in terms of 
reduced sets of data as will now be shown” (McQuitty, 1965). 


The Isolation of Multiple, Independent Types 


In order to illustrate Multiple Rank Order Typal Analysis for 
the Isolation of Independent Types we applied it to the classifi- 
cation of eight industrial companies in terms of union-manage- 
ment characteristics. These data were again chosen for two 
reasons: (a) they are a small set of highly structured data which 
can be economically analyzed and displayed, and (b) the results 
can be compared concisely with those of earlier analyses of the 
same data by other methods. 

The matrix of interassociation between the eight companies is 
shown in Table 5 below. 


TABLE 5 
Agreement Scores between Companies* 


A B Cc D E F G H 


Honsggawe 
m 
© 
= 
Se 
w 
& 
o 
re) 
= 
© 
ey 
D 
m 
are 
m 
jar 


* Data from McQuitty, 1954 


The first step of both the Intersecting and the Independent 
Versions is to apply a Rank Order Typal Analysis (McQuitty, 
1963) as outlined above. 

The Rank Order Typal Analysis yielded the following results: 
Companies A and B, two construction companies, first joined to 
form a type, followed in order by C and D, two trucking com- 
panies; G and H, two garment manufacturing companies; and then 
by E and F, a grain processing company and a products manu- 
facturing company. In the final operation, two types, AB and CD, 
joined to form a new type, ABCD; no other combinations of com- 
panies qualified as a type under our above definition. 

The steps unique to the Independent Version were applied as 
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follows: the most inclusive types were defined. They are the mini- 
mum number of types required to include all of the companies 
which classify into types at this stage; in this case they are Types 
ABCD, EF, and GH; Types AB and CD were omitted because 
they are included in Type ABCD. For each inclusive type, the 
characteristics common to the members of that type were first 
specified and then they were subtracted out of the original de- 
scription of each of the individual companies. For example, Com- 
panies A, B, C, and D agreed on 13 of the 32 original, dichotomous 
items, and the subtraction left each of these four companies de- 
scribed in terms of but 19 items. As a result, each of these com- 
panies could no longer agree with any company in terms of the 
items common to Type ABCD. 

Analogously, the common characteristics were selected out of 
the description of the two companies for each of the other two 
inclusive types, Types EF and GH. 

A new matrix of agreement scores was prepared between all of 
the 8 companies in terms of the items which remained for each 
individual company. 

A Rank Order Typal Analysis was performed on this matrix. 
It yielded Types AB and CD; no other combinations qualified 
as types. These Types, AB and CD, had already appeared on the 
way to yielding Type ABCD. 

The method proceeds with Types AB and CD, just as it would 
with any other “inclusive” types (as defined above). 

The common characteristics of each Type AB and Type CD 
were subtracted from each A and B, on the one hand, and C and 
D, on the other. 

A new matrix of agreement scores was formed, just the same as 
the last one except for the omissions of items in Companies A, B, 
°C, and D as specified in the above paragraph. 

The last matrix was analyzed by Rank Order Typal Analysis and 
failed to yield any types. The analysis was thus completed. 


Versions in the Search for Independent Types 


There are versions in the method of searching for independent 
types. We could have used first-order types rather than inclusive 
types in the operation where we eliminate characteristics common 
to the members of the types isolated. 
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In this study, the first-order types of the first application of 
Rank Order Typal Analysis are Types AB, CD, EF, and GH; they 
are the lowest order of types within the typology which includes 
every company with one or more other companies and does not 
include any company more than once. 

If we had used the first-order types, AB, CD, EF, and GH, 
rather than the inclusive types, ABCD, EF, and GH, the first 
elimination would have produced the final matrix. 

The above point is clarified by a brief review of steps in the 
present analysis. We eliminated what was common to the members 
of Type ABCD. The subsequent analysis produced again Types 
AB and CD, this time as inclusive types, and we then eliminated 
what was still common on the one hand to the members of Type 
AB and on the other hand what was still common to the members 
of Type CD. 

The use of first-order types would have eliminated one step in 
the present study. This result does not ensue because of the 
nature of the method alone, but because of the nature of the 
method in relation to the data. 

The use of inclusive types as contrasted with first-order types 
provides for the isolation of a typology (or typologies) which 
might otherwise remain hidden. However, only the inclusive types 
of the first analysis would of necessity be independent of the types 
of the subsequent analyses; the first-order types of the first analysis 
would likely intersect with types of the subsequent analyses; they 
would not, however, of necessity intersect with them. 

The use of first-order types would require that all types of the first 
analysis be independent of all types of subsequent analyses. 

The two analyses yielded the results shown in Figures 1 and 2. 

Figure 1 reports the exact number of characteristics that the + 
members of each type have in common. Furthermore, it reveals the 


Types and Agreements ABCD = 13 


Types and Agreements K se 26 a He 21 
Bo, a8 /\ G H E E 


Individual Companies 
Figure 1. First Analysis 
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cD = 13 


Individual Companies A B i E F G H 


Figure 2. Second Analysis 


Types and Agreements AB 4 16 


number that the members of each AB and CD have in common 


over and above those which all four companies have in common 8&8 
members of Type ABCD; they are 16 and 13, respectively (29- 
13—16 for Type AB and 26-13=13 for Type CD). This latter 
fact is reported also in Figure 2, but Figure 2 still adds information 
not revealed by Figure 1; it shows that the data of this study re- 
veal only one system of independent types, in terms of the strin- 

gent definition of types used in the above analysis. : 


Interpretation 


Tt is helpful to contrast the above analysis in search of independ- 
ent types with those of the same data in search of intersecting types 
(McQuitty, 1965). 

In addition to the typology shown in Figure 1, the search for 
intersecting types yielded the typology of Figure 3. 


ARÇD EFGH 


elk s SJ 


Figure 3. A Typology for a Search for Intersecting Types 


The results of Figure 3 were obtained if we performed a Rank” 
Order Typal Analysis using the characteristics of either Type CD, 
EF, or GH, exclusively, but not if we used either all of the char- — 
acteristics of the original study or those of Type AB exclusively. 
In these latter two cases, we obtained the typology of Figure 1. ; 

The agreement scores of the types in Figure 3 are not reported — 
because they vary depending on whether the set of characteristics 
used was from Type CD, EF, or GH. j 

The present set of data reflects two similar configurations of în- 
tersecting typologies but only one system of independent types: 
The substantive implications of this very limited study are that 
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various pressures do in fact act on institutions to yield more than 
one typology and that the pressures interact to yield intersecting 
rather than independent types. 

The substantive implications of this study need to be investi- 
gated further with much more comprehensive sets of data in rela- 
tion to people, institutions, and other objects in many cultures. 
The contributions of this and related papers are in the methodol- 
ogies which they provide for the substantive studies. 


Summary 


This paper assumes that various sets of pressures act on in- 
dividuals, (institutions and other objects) as they develop. As a 
consequence, each of the several individuals tends to develop more 
than one type, and a sample of individuals reflects more than one 
typology. The multiple existence of types obscures them from 
unaided observation. 

Quantitative methods are needed to help separate out and dis- 
play the various types and typologies. In developing such methods, 
it is possible to assume either that the types (and the typologies) 
are independent, or that they are intersecting. 

In this paper, it is assumed that the multiple types (and multiple 
typologies) are independent, and a quantitative method to assist in 
isolating them is developed and illustrated with empirical data. 
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KUDER-RICHARDSON RELIABILITIES OF 
CLASSROOM TESTS 


EDWARD E. CURETON 
University of Tennessee 


Crassroom tests often contain substantial numbers of items so 
easy that everyone or almost everyone gets them right. The pur- 
pose of the present note is to show the effects of this situation on 
estimates of the reliabilities of such tests by the Kuder-Richardson 
formulas 20 and 21. Ordinarily the too-easy items are merely con- 
sidered “dead wood”: they take up the time of the examinees but 
are not supposed to contribute either positively or negatively to the 
reliability of the test. 

We will use an extreme case as an example. Suppose we have 
initially a test of k “good” items; “good” in the sense that they all 
measure the same trait or ability, apart from error, and that every 
one of them has positive and significant discrimination. To this 
test we add k additional items, all so easy that every examinee gives 
the right answer to every one of them. We assume that every item 
is scored 1 for right and 0 for wrong. For the initial set, we have 


k 
xen = Aq [1 3h 


where § is a summation over the k items and o;? is the variance of 
the total scores. The added items will not change the variance of 
the total scores, and pq will equal 0 for every one of them, so the 
expression in brackets will remain unchanged. For the augmented 
test we then have, 


_ 2&k _|, _ Spa 
Kta = [1 sa). 
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The estimate of reliability will be reduced slightly but not seri- 
ously. 

Thus, if k = 25, k/(k—1) = 1.04, and 2k/(2k—1) = 1.02. 

Also for the initial set of k items, 


ze- hli 2564] 


Again the added items will not change o;?, but X will be increased 
by k. For the augmented test we than have, 


Kita = 3 Sewers | 
as a 
ad (eS ef Sua aa 
2—1 gr. 
The added term is intrinsically positive, so the reliability of the 
augmented test will be still further underestimated by KR21ı. Thus 
if k = 25, X = 15, and o? = 4, KRa will give .65 for the initial 
test and .51 for the augmented test. 

Tt follows that KRə should not be recommended as a short 
formula for estimating the reliabilities of classroom tests, or in gen- 
eral, of any tests whose nonfunctional items have not been elim- 
inated in advance by item analysis. 


ga 
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A COMPARISON OF 57 MMPI SCALES AND 57 
EXPERIMENTAL SCALES MATCHED WITH THE MMPI 
SCALES IN TERMS OF ITEM SOCIAL DESIRABILITY 
SCALE VALUES AND PROBABILITIES OF 
ENDORSEMENT! 


ALLEN L. EDWARDS? 
University of Washington 


Ir has been shown (Edwards and Diers, 1962; Edwards, Diers, 
and Walker, 1962; Edwards and Heathers, 1962) that first factor 
loadings of Minnesota Multiphasic Personality Inventory (MMPI) 
scales are highly correlated with the zero order correlations of the 
scales with Edwards’ (1957) Social Desirability (SD) scale, The 
SD scale was designed to measure a general personality trait, 
namely, the tendency to give socially desirable responses in self- 
description under the standard instructions ordinarily used in ad- 
ministering personality scales and inventories. A socially desirable 
response is defined as a True response to an item with a socially 
desirable scale value or as a False response to an item with a so- 
cially undesirable scale value. All of the items in the SD scale are 
keyed for socially desirable responses. 

If the trait keyed responses to the items in a trait scale are also 
socially desirable responses, then the trait keying and the social 
desirability keying are completely confounded. A high score on 
the trait scale may be obtained by a subject who responds to the 
items in terms of the trait which the scale was designed to measure, 

1 This research was supported in part by Research Grant MH 04075 from 
the National Institute of Mental Health, United States Public Health Service. 

2T am indebted to Dr. Benard H. Taylor for assistance in the collection of 
the test records used in this study; to Dr. James A. Walsh for selecting the 


items in the Experimental Personality Inventory; and to Dr. Walsh and Alan 
J. Klockars for supervising the runs on the computer. 
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but a high score may also be obtained by a subject who has a 
strong tendency to give socially desirable responses. If the tend- 
ency to give socially desirable responses operates with respect to 
personality items in general and not just with respect to the items 
in the SD scale, then scores on trait scales which have a large 
proportion of items keyed for socially desirable responses should 
be positively correlated with scores on the SD scale. Similar con- 
siderations apply to trait scales which have a small proportion of 
items keyed for socially desirable responses and, consequently, a 
large proportion of items keyed for socially undesirable responses. 
If the tendency to give socially desirable responses is operating, 
then, in this instance, high scores on the SD scale should be as- 
sociated with low scores on the trait scale and the correlation 
between the trait scale and the SD scale should be negative. The 
fact that both the correlations of MMPI scales with the SD scale 
and the first factor loadings of the scales have been found to be 
highly correlated with the proportion of items keyed for socially 
desirable responses in the scales (Edwards, 1961; Edwards and 
Diers, 1962; Edwards, Diers, and Walker, 1962; Edwards and 
Heathers, 1962) offers fairly convincing evidence that responses 
to items in MMPI scales are influenced by social desirability tend- 
encies. 

The present study was undertaken to determine whether the re- 
lationship observed between the proportion of items keyed for so- 
cially desirable responses in MMPI scales and the correlations of 
these scales with the SD scale is restricted to scales composed of 
MMPI items. More specifically, if we build experimental scales 
consisting of non-MMPI items but such that each experimental 
scale has certain structural properties in common with a corres- 
ponding MMPI scale, will the correlations of the experimental 
scales with the SD scale be proportional to the correlations of the 
corresponding MMPI scales with the SD scale? Furthermore, will 
the internal consistency of the experimental scale be proportional 
to the internal consistency of the corresponding MMPI scale? 
And will the correlation between the experimental scale and the 
corresponding MMPI scale be related to the proportion of items 
keyed for socially desirable responses in the scales? 

MMPI scales have been constructed by a variety of techniques. 
Some scales were developed by finding those items in the MMPI 
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which differentiated between a criterion and a control group. 
Others were constructed using some form of internal consistency 
analysis. Still others were developed on a rational or construct 
basis. Presumably, each MMPI scale was developed to measure 
some trait of interest and, presumably, each scale has some degree 
of predictive, content, construct, or concurrent validity. The ex- 
perimental scales constructed for the present study, on the other 
hand, cannot be said to possess any validity of any kind in any 
degree. The basis for this statement is that the experimental scales 
have never been scored before and the items in each of the scales 
were not selected on the basis of content or by means of any kind 
of internal consistency analysis. 

The experimental scales, however, do have some structural 
properties in common with MMPI scales. First of all, for each 
MMPI scale there is a corresponding experimental scale scored by 
the same scoring stencil as the MMPI scale. The corresponding 
MMPI and experimental scales have the following properties in 
common: (1) each scale contains the same number of items; (2) 
the location of the items in the experimental scale in a set of 566 
items is the same as the location of the items in the MMPI scale in 
a set of 566 items; (3) the corresponding items in the experimental 
scale and the MMPI scale have approximately the same social de- 
sirability scale values and also approximately the same probabili- 
ties of being answered True; and (4) the keying of the items in the 
experimental scale is exactly the same as the trait keying of the 
items in the MMPI scale. Thus, it follows that both scales have 
the same proportion of items keyed for socially desirable responses 
and that the intensity of the social desirability keying of the items 
in the corresponding experimental and MMPI scale is approxi- 
mately the same. 


Method and Procedure 


Messick and Jackson (1961) have provided social desirability 
scale values (SDSVs), based upon the judgments of college stu- 
dents, for each of the 566 items in the MMPI. Goldberg and Rorer 
(1963) give the probabilities of a True response, P(T), based upon 
the responses of college students, for each of the MMPI items. 
Edwards (1963) has obtained SDSVs and P(T)s for each of the 
items in his pool of 2,824 experimental personality statements. The 
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2,824 experimental items were sorted according to their SDSVs. 
Then, using the SDSV of each item in the MMPI, a corresponding 
experimental item was selected from the pool of 2,824 items with 
approximately the same SDSV and P(T) as the MMPI item and 
without regard to the content of the experimental item. For ex- 
ample, the first three items in the MMPI are: (1) I like mechanics 
magazines; (2) I have a good appetite; and (3) I wake up fresh 
and rested most mornings. The corresponding items in the Experi- 
mental Personality Inventory (EPI) are: (1) I value logic above 
personal feelings; (2) I am not resistant to new ideas; and (8) I will 
keep at a problem long after others have given it up as hopeless. The 
SDSVs and P(T)s of these paired items are: 


MMPI EPI 
Item SDSV P(T) SDSV P(T) 
1, 5.48 65 5.60 .58 
2. 6.78 96 6.78 -96 
3 7.75 46 7.65 -46 


In the same manner, each item in the EPI was matched with a cor- 
responding item in the MMPI in terms of SDSV and P(T). No 
consideration was given to the content of the items selected for the 
EPI. 

The MMPI was administered to a group of male college stu- 
dents. Approximately a week later the EPI was administered. Test 
records for both the MMPI and the EPI were obtained from 138 
male students. Using the scoring stencils for each of 57 MMPI 
scales, scores on the 57 MMPI scales were obtained for each stu- 
dent, based upon his responses to the MMPI items.® The same scor- 
ing stencils were then applied to the student’s EPI answer sheet 
to obtain corresponding scores on the 57 EPI scales, based upon 
his responses to the EPI items. Scores on the 114 scales were in- 
tercorrelated and factor analyzed by the method of principal com- 
ponents. Two factors were extracted. The first factor accounted 
for 37 percent of the total variance and the second factor ac- 
counted for 10 percent of the total variance. These percentages are 
comparable to those obtained when only the MMPI scales are fac- 


3 The code names of the MMPI scales are as follows: A, Ac, Ad, Ai, At, B, 
Ca, Cn, D, Dn, Do, D-0, D-8, Dy, Eo, Es, F, Fm, Ho, Hs, Hy, Hy-O, Hy-S, 
Ie, Im, K, L, Lp, Ma, Ma-O, Ma-S, Mj-m, Mp, Ne, No, Nu, Or, Pa, Pa-O, 
Pa-S, Pd, Pd-O, Pd-S, Pn, Pr, Pt, Pv, R, Re, Rp, Sc, Si, SD, Sp, St, Tt, Sd. 
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tor analyzed (Edwards, Diers, and Walker, 1962). Of primary in- 
terest are the first factor loadings of the scales, since it is the first 
factor of the MMPI which has been interpreted as a social de- 
sirability factor. 

The original SD scale is composed of 39 MMPI items and this 
scale will be referred to as SD,. The corresponding EPI SD scale 
will be referred to as SD2. The correlations of the MMPI scales 
with scores on both SD; and SDz were obtained. The correlations 
between the MMPI scales and SD; cannot be attributed to item 
overlap because SD2 contains no items in common with any of the 
MMPI scales. The corresponding correlations between the EPI 
scales with SD, and SD, were obtained. In this instance, the corre- 
lations between the EPI scales and SD, cannot be attributed to 
item overlap because SD, contains no items in common with any of 
the EPI scales. 

The first factor loadings of each of the EPI and MMPI scales 
and the correlations of each of the scales with SD, and SD3 repre- 
sent dependent variables of interest. A common property of each of 
the corresponding EPI and MMPI scales is the proportion of 
items keyed for socially desirable responses and this is an inde- 
pendent variable of interest. We thus obtained a matrix with 57 
rows, corresponding to the 57 scales, and 7 columns, corresponding 
to the first factor loadings of MMPI scales, the first factor load- 
ings of EPI scales, the correlations of the MMPI scales with SD: 
and with SDs, the correlations of the EPI scales with SD, and 8D», 
and the proportion of items in the scales keyed for socially desirable 
responses. Correlations were then obtained between each of the 
column variables. 

All of the variables described above, with the exception of the 
proportion of items keyed for socially desirable responses in the 
scales, are signed variables, that is, the first factor loadings of both 
EPI and MMPI scales are in some cases positive and in other cases 
negative and this is true also of the correlations of the scales with 
SD, and SDs. All of these dependent signed variables, according to 
social desirability considerations, should be linearly related to one 
another and to the proportion of items keyed for socially desirable 
responses. There are other variables of interest however, which, 
according to social desirability considerations, should be non-lin- 
early related to the signed first factor loadings and signed corre- 
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lations of the scales with the two SD scales but which should be 
linearly related to the absolute values of these variables. 

For example, according to social desirability considerations, if we 
obtain Kuder-Richardson Formula 21 (K-R 21) values for each of 
the scales and plot these values against: (1) the signed first factor 
loadings, (2) the signed correlations of the scales with the two SD 
scales, or (3) the proportion of items keyed for socially desirable 
responses, we should expect to find higher K-R 21 values for scales 
which have values at both extremes of these three variables than for 
scales which have values in between the two extremes. Scales with 
middle values are scales which have: (1) low loadings on the first 
factor; (2) low correlations with the SD scales; and (3) a balance 
in their social desirability keying. Scales with any of these prop- 
erties should have relatively lower K-R 21 values than scales which 
do not. 

Similar considerations apply to the correlations between corre- 
sponding EPI and MMPI scales. All of these correlations are pre- 
dicted to be positive in sign. But high values of the correlations 
should be associated with scales which have either a large or a small 
proportion of items keyed for socially desirable responses and low 
values should be associated with scales which have a better balance 
in their social desirability keying. Thus, it follows that high corre- 
lations between the EPI and MMPI scales should also be associated 
with scales which have either high positive or high negative first 
factor loadings and which have either high positive or high nega- 
tive correlations with the SD scale. 

For the reasons cited above, intercorrelations were also obtained 
between the K-R 21 values of the MMPI and EPI scales, the ab- 
solute values of the first factor loadings of the EPI and MMPI 
scales, the absolute values of the correlations of the scales with the 
two SD scales, the product of the first factor loadings of corre- 
sponding EPI and MMPI scales, and the correlations between cor- 
responding EPI and MMPI scales. 


Results and Discussion 


Table 1 gives the intercorrelations between the signed first factor 
loadings of the EPI and MMPI scales, the signed correlations of 
the EPI and MMPI scales with the two SD scales, and the propor- 
tion of items keyed for socially desirable responses in the scales. It 


ALLEN L. EDWARDS 21 


is obvious that the first factor loadings of the MMPI scales could 
be predicted quite accurately (r = .98) from the factor loadings of 
the EPI scales.t It is also clear that the correlations of the EPI 
scales with SD, are proportional to the corresponding correlations 
of MMPI scales with SD,, despite the fact that SD, contains no 
items in common with any of the EPI scales. Similarly, the correla- 
tions of the MMPI scales with SD2 are proportional to the corre- 
sponding correlations of EPI scales with SDz, despite the fact that 
SD» contains no items in common with any of the MMPI scales. 
Of interest also is the fact that the SD1 correlations with the EPI 
scales are proportional to the SD2 correlations with the MMPI 
scales. 


TABLE 1* 


Intercorrelations between the Reflected First Factor Loadings of 57 MMPI 
Scales (I), the Reflected First Factor Loadings of 57 EPI Scales (I'), 
the Correlations of the MMPI and EPI Scales with Two Social 
Desirability Scales (SD: and SD2), and the Proportion of 
Items Keyed for Socially Desirable Responses in the 
Scales, P(SD). Means and Standard Deviations 
Are Given in the Last Two Rows 


MMPI EPI MMPI EPI MMPI EPI 
I iv SD: SD: SD: SD: P(SD) 


Se eee 
I _ -98 -99 -97 -99 -93 -90 


p -= .98 .99 .97 .94 -90 
MMPI: SD: = -97 -99 .92 .91 
EPI: SD: — -96 .94 -90 
MMPI: SD2 = +92 93 
EPI: SDs = 87 
x =—12 —18° =.138 =—.11° —11 —.12 39 
8 -63 .58 .58 .44 -50 54 33 


Se a i 
* The SD scale and other scales with a large proportion of items keyed for socially desirable 
responses have negative loadings on the unreflected first factor. The factor loadings were reflected 
so that these scales would have positive loadings on the reflected factor and scales with a small 
proportion of items keyed for socially desirable responses would have negative loadings. 


The last column of Table 1 gives the correlations between the 
proportion of items keyed for socially desirable responses in the 
EPI and MMPI scales and the other dependent variables. No mat- 
ter whether we look at the first factor loadings of the MMPI and 
EPI scales or the correlations of the MMPI and EPI scales with 
either of the two SD scales, the proportion of items keyed for 


«The second factor loadings of MMPI scales correlated .67 with the cor- 
Tesponding second factor loadings EPI seal: 
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socially desirable responses in the scales is a fairly accurate predic- 
tor of the factor loadings and of the correlations of the scales 
with the SD scales. 

The most parsimonious explanation of the correlations given in 
Table 1 would appear to be in terms of the fact that corresponding 
MMPI and EPI scales have exactly the same proportion of items 
keyed for socially desirable responses. When this proportion is 
large in an MMPI scale and, consequently, in an EPI scale, then 
high scorers on the SD scales also tend to obtain high scores on the 
EPI and MMPI scales and, as a result, the correlations of both the 
EPI and MMPI scales tend to be positively correlated with the 
SD scales. When the proportion of items keyed for socially de- 
sirable responses in an MMPI scale is small, then high scorers on 
the SD scales tend to obtain low scores on both the MMPI and 
EPI scales and the correlations of the MMPI and EPI scales with 
the SD scale both tend to be negative. The correlations given in 
Table 1 can thus be accounted for in terms of a common trait, the 
tendency to give socially desirable responses in self-description and 
by the fact that some MMPI and corresponding EPI scales are 
better measures of this trait than other MMPI and corresponding 
EPI scales by virtue of the imbalance in the social desirability key- 
ing of the items in the scales. 


TABLE 2 


Frequency Distribution of the Obtained Correlation Coefficients between 
57 Paired MMPI and EPI Scales (f) and the Correlations 
Corrected for Attenuation (f') 


Correlations f 


-90-.99 
-80-.89 
-70-.79 
-60-.69 
-50-.59 1 
-40-.49 
-30-.39 1 
-20-.29 
-10-.19 


pete 


NNR OWA 
m. 
I at o 0 0 H ORD 


Table 2 shows the frequency distribution of the correlation co- 
efficients between the paired MMPI and EPI scales. Test-retest re- 
liability coefficients were available for each of the 57 MMPI scales 


f 
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based upon responses of another sample of 95 male college stu- 
dents.’ Test-retest reliability coefficients were not available for the 
57 MMPI scales. However, assuming that the coefficients for the 
MMPI scales represent upper-bounds for the corresponding EPI 
scales, the correlations between the paired MMPI and EPI scales 
were corrected for attenuation using the reliability coefficients for 
the MMPI scales. The frequency distribution of the corrected 
correlations is given in the last column of Table 2. 

In terms of social desirability considerations we are interested in 
accounting for the variation in the magnitude of the correlation 
coefficients between paired MMPI and EPI scales. That is, we 
would like to understand why some pairs of MMPI and EPI scales 
have relatively high correlations and other pairs have relatively low 
correlations. Table 3 shows the degree to which the correlations be- 
tween the paired MMPI and EPI scales are related to the K-R 21 
values of the MMPI and EPI scales, the absolute values of the first 
factor loadings of the MMPI and EPI scales, the absolute values of 
the correlations of the MMPI and EPI scales with the SD scales, 
and the product of the signed first factor loadings of the paired 
MMPI and EPI scales. Table 3 also gives the intercorrelations of 
these variables. 

Tt has already been shown in Table 1 that the first factor loadings 
of MMPI and EPI scales are highly correlated. Thus, when an 
MMPI scale has a high positive loading on the first factor, so also 
does the corresponding EPI scale and the product of the factor 
loadings is also high and positive. Similarly, when an MMPI scale 
has a high negative loading on the first factor, so also does the cor- 
responding EPI scale and the product of their first factor loadings 
is also high and positive. Table 3 shows that the products of the 
first factor loadings of corresponding MMPI and EPI scales cor- 
relate .85 with the observed correlations between the corresponding 
MMPI and EPI scales. Table 3 also shows that the higher the abso- 
lute value of the correlation of an MMPI scale with the SD scales, 
the higher the correlation of the MMPI scale with the corresponding 
EPI scale. In other words, when an MMPI scale has either a high 
positive or a high negative correlation with the SD scales it, in turn, 


5I am indebted to Lewis R. Goldberg and Leonard G. Rorer of the Oregon 
Research Institute for calculating and making available to me the test-retest 
coefficients of the MMPI scales. 
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TABLE 3 


Intercorrelations between the K-R 21 Values of 5 MMPI and 57 EPI Scales, the A 
Values of the First Factor Loadings of the Scales, the Absolute Values of the 
Correlations of the Scales with Two SD Scales, the Product of the 
Signed First Factor Loadings of the Scales, and the 
Correlations between Corresponding 
MMPI and EPI Scales 


MMPI EPI MMPI EPI MMPI EPI MMPI EPI i 
K-R21 K-R21 I Y SD, SD: SD: SDsI XI’ repr 


tends to have a relatively high positive correlation with its EPI 
counterpart. 

It has already been shown that the first factor loadings of both 
MMPI and EPI scales are related to the proportion of items keyed 
for socially desirable responses in the scales, It has also been shown 
that MMPI scales which have high positive correlations with the 
SD scales are those which also have a large proportion of items 
keyed for socially desirable responses and that MMPI scales which 
have high negative correlations with the SD scales are those which 
also have a large proportion of items keyed for socially undesirable 
responses. Thus, the magnitude of the correlation between paired 
EPI and MMPI scales is related to the imbalance in the social de- — 
sirability keying of the scales. MMPI scales which have either a 
large proportion of items keyed for socially desirable responses or 
a large proportion of items keyed for socially undesirable responses 
tend to have higher correlations with their EPI counterparts than 
MMPI scales which have a better balance in their social desirability 
keying. 

Reasons have been presented elsewhere (Edwards, 1964; Ed- 
wards, Walsh, and Diers, 1963) to show why, if the tendency to 
give socially desirable responses is operating, personality scales in 
which the trait keyed responses are also consistently socially de- 
sirable or consistently socially undesirable responses should have a 
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higher degree of internal consistency than scales which are more 
balanced in their social desirability keying. The results reported in 
Table 3 are in accord with this prediction both with respect to the 
MMPI and EPI scales. Both MMPI and EPI scales which have high 
absolute correlations with either of the two SD scales tend to have 
a higher degree of internal consistency, as measured by the K-R 
21 values of the scales, than scales which have low correlations 
with the SD scales, Since the important characteristic of corre- 
sponding MMPI and EPI scales which have high absolute correla- 
tions with the SD scales is that they have either a large proportion 
of items keyed for socially desirable responses or a large proportion 
of items keyed for socially undesirable responses, it follows that 
the internal consistency of the scales is related to the imbalance in 
the social desirability keying of the items in the scales, It may also 
be noted that the K-R 21 values of the EPI scales are related to the 
K-R 21 values of the corresponding MMPI scales, the correlation 
between the paired K-R 21 values being .78, and that, in general, 
the higher the K-R 21 value of an MMPI scale, the higher its cor- 
relation with the corresponding EPI scale. 

The relationships found between the correlations of MMPI 
scales with the SD scale, the first factor loadings of the MMPI 
scales, the K-R 21 values of the MMPI scales, and the proportion of 
items keyed for socially desirable responses in the MMPI scales, in 
the present study, are in accord with the results of previous studies. 
Of greater significance, however, is the fact that essentially the 
same relationships between these variables are obtained when the 
scales involved are the EPI scales consisting of non-MMPI items. 

In view of the fact that each of the items in the EPI was selected 
solely on the basis of the SDSV and P(T) of a corresponding item 
in the MMPI, it does not seem reasonable to believe that the items 
in each of the 57 EPI scales are similar in content to the content of 
the items in each of the corresponding 57 MMPI scales. Tf corre- 
sponding EPI and MMPI scales do not have a common content, 
they do have in common the same intensity of social desirability 
keying and the same proportion of items keyed for socially de- 
sirable responses and the latter variable has been shown to be a 
quite accurate predictor of the relationship between the correlation 
of MMPI and EPI scales with the SD scales and the first factor load- 
ings of the MMPI and EPI scales. 
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Summary 


An Experimental Personality Inventory (EPI) was developed in 
which each item was selected to match a corresponding item in the 
Minnesota Multiphasic Personality Inventory (MMPI) in terms of 
the social desirability scale value of the MMPI item and the prob- 
ability of a True response to the MMPI item. The MMPI and EPI 
were given to 138 males and both inventories were scored for 57 
MMPI scales. The first factor loadings and the correlations of the 
MMPI and EPI scales with two different SD scales were found to 
be highly correlated with the proportion of items keyed for so- 
cially desirable responses in the scales. The magnitude of the cor- 
relations between paired MMPI and EPI scales was found to be 
related to the imbalance in the social desirability keying of the 
items in the scales. MMPI scales with high Kuder-Richardson 
Formula 21 (K-R 21) values were found to have higher correla- 
tions with their EPI counterparts than MMPI scales with low K-R 
21 values. 
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Introduction 


Tun growth of various health related professions has presented 
an attending problem in the guidance and selection of students 
planning to enter one of these professions, such as medical tech- 
nology, occupational therapy, and physical therapy. The concen- 
trated educational programs for these professions begin, for the 
most part, at the junior level in the university. There are, however, 
some rather general, introductory courses that cut across all of the 
professions and are required at the freshman and sophomore levels. 
It is becoming increasingly important to study characteristics and 
attributes of these beginning students for examining problems of 
differential program selection, dropouts, laboratory and clinical 
performance, and related aspects of students success. 

The College of Health Related Professions (HRP) at the Uni- 
versity of Florida represents a constellation of educational pro- 
grams for the health related professions. A majority of the under- 
graduate students are females who, at the freshman and sophomore 
level, plan to enter occupational therapy, physical therapy, or med- 
ical technology. The present study is an investigation of the per- 
_ 

1 This research was supported in part by a research grant from the Voca- 


tional Rehabilitation Administration, Department of Health, Education, and 
Welfare, Washington, D. C. 


29 


30 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


sonality structure of these students for comparisons with other 
students and for use in future research, 


Method 
Sample 


A sample of 168 female students was used in this study. All of 
the students were freshmen and sophomores in the HRP College, 
Moreover, all the students were enrolled in the basic course, “In- 


troduction to Health Related Services,” during the years 1962 to 
1964, 


Variables? 


The ten basic clinical scales, excluding the four validity scales? 
of the Minnesota Multiphasic Personality Inventory (MMPI) were 
used in the study. The ten scales are as follows: Hypochondriasis 
(Hs), Depression (D), Hysteria (Hy), Psychopathic Deviate (Pd), 
Masculinity-Feminity (M-F), Paranoia (Pa), Psychasthenia (Pt), 
Schizophrenia (Sc), Manic-Depressive (Ma), and Introversion 
(Si). The Hs, Pd, Pt, Sc, and Ma scales were used in K-corrected 
form, a usual scoring procedure in use of the MMPI and consid- 


ered to have negligible effects on the interrelationships among the 
Personality scales. 


Analysis* 


Scores for the 168 students on the 10 scales were intercorrelated 
by the product-moment method. The 10 X 10 correlation matrix, 
with squared multiple correlations in the principal diagonal, was 
factored by the principal component method, and the resulting 
factor structure was rotated by the normalized varimax routine 
(Kaiser, 1958), 

The main interest in the study is the interrelationship among 
personality variables, and not necessarily the interrelationships 


— 

2Part of the data for this study was collected under NIMH Project Grant, 
380, the Public Mental Health Methods in a University. T 

2 Only seven cases exceeded a T-score above 70 on the K scale, and none 
saeara eon 70 on the F and L scales. 

*4The analyses of data for this study were made possible through the use of 
personnel and facilities at the Computer Center of the Wale of Florida 
and the Computer Center of the University of Georgia. 
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among scoring artifacts in the MMPI scales. Items overlap among 
the scales, for instance, biases the data such that one could get a set 
of intercorrelations (e.g., Dahlstrom and Welsh, 1960, p. 82) among 
the scales if responses were made from a table of random numbers; 
we shall, therefore, refer to these as random correlations. Since the 
random correlations are not equal across all pairs of scales, there 
is no single base from which obtained correlations can be com- 
pared, and there is no obvious immediate way to estimate the im- 
pact of the random correlations on the underlying factor structure 
of the clinical battery. There are several indirect ways to estimate 
the impact, such as the one by Welsh (1952) wherein he removed 
some of the overlapping items in an attempt to factor somewhat 
more “pure” scales, but this method leaves very few items in some 
of the scales which tends to reduce standard deviations, reliability, 
and possible correlational relationships obtainable with the scales. 
Another method, which will be used herein, is to add or subtract, 
as the case may be, the random correlations from the obtained 
ones; squared correlations, however, will be used directly in the 
adding and subtracting process since they represent predictable 
variance estimates. The resulting “corrected” correlation matrix, 
with the original squared multiple correlations throughout the 
principal diagonal, was factored and factor-rotated by similar com- 
puter routines, and the resulting rotated factor structures were 
compared by the canonical correlation method. (e.g., Anderson, 
In Press; Kendall, 1957, Rao, 1952). This method allows for assess- 
ment of the impact of the random correlations on the underlying 
factor structure with consequent confidences in the evaluation of a 
personality structure obtainable with the MMPI. 


Results 


The obtained means, standard deviations, and inter-correlations, 
with squared multiple correlations in the principal diagonal, for the 
10 MMPI scales are presented in Table 1. The means and standard 
deviations, presented in T-score units, look strikingly similar to 
the standardization norms for the MMPI and suggest confidence in 
the comparability of this sample to the standardization sample. It 
is the underlying factor structure, however, that will be of greatest 
interest. 

For ease of comparisons, the corrected correlations are pre- 


* Decimals omitted from correlations. 
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TABLE 1 
Intercorrelations, Means, and Standard Deviations of the MMPI Scales* 


Scales 


Hs D Hy Pd MF Pa Pt Sc Ma _ Si_ Means 


7219 5496 8154 4675 3303 4619 6206 5880 2504 3430 50.3155 
5496 6808 4639 3674 1295 5868 7018 5362 0340 7141 50.2679 
8154 4639 7302 5576 3275 5275 5813 5876 2790 2575 55.0357 9.6 
4675 8674 5576 4883 2442 4507 5334 6224 4283 2076 55.0476 11.20 


3803 1295 3275 2442 1883 1246 2419 3475 2759 0818 48.0655 10.15 
4619 5868 5275 4507 1246 50797 6176 5854 2199 4971 52.9762 9.70 
6206 7018 5813 5334 2419 6176 7434 7609 3350 6267 54.2738 9,62 


5880 5362 5876 6224 3475 5854 7609 6954 4533 4635 53.8095 9 
2504 0340 2790 4283 2759 2199 3350 4533 3390 0146 55.1667 
3430 2575 2076 0818 4971 6267 50.5179 


sented in Table 2. The major reductions of correlations are be- 
tween Hy and Hs from .82 to .67, for Pt and D from .70 to .65, be- 
tween Pt and Sc from .76 to .71, and for Ma and D, from .03 to .18. 
This matrix also contains the original squared multiple correlations 
throughout the principal diagonal. 

The correlation matrices in Tables 1 and 2 were factor analyzed 
separately by the principal component method. In accordance with 
the computer program cirterion, extracting principal axes with 
eigenvalues greater than zero, four factors, as shown in Table 3, 
were required to trace the matrix of uncorrected correlations in 
Table 1, while six were required for the corrected correlations. 
Actually, Factor IV in Table 3 and Factors IV through VI in Table 
4 appear to be residual factors. 


TABLE 2 
Corrected Intercorrelations of the MMPI Scales* 
Scales 
Scales Hs D Hy Pd MF Pa Pt Sc Ma “Si 
Hs 7219 5193 6733 4671 3303 4609 6186 5825 2504 3424 
D 5193 6808 4084 3397 1261 5854 6518 5268 0600 7084 
Hy 6733 4084 7302 5277 3219 5213 5666 5751 2724 2728 
Pd 4671 3397 5277 4883 2450 4132 5197 6015 4164 2086 
M-F 3303 1261 3219 2450 1883 1179 2411 3463 2713 0795 
Pa 4609 5854 5213 4132 1179 5079 6110 5383 2143 5007 
Pt 6186 6518 5666 5197 2411 6110 7434 7114 3296 6152 
Se 5825 5268 5751 6015 3463 5383 7114 6954 4161 4608 
Ma 2504 0600 2724 4164 2713 2143 3296 4161 3390 0717 
Si 3424 7084 2728 2086 0795 5007 6152 4608 0717 5930 


* Decimals omitted. 
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There is a general reduction in the communalities from the 
analysis in Table 3 to that in Table 4. As might be expected, the 


TABLE 3 
Factor Matriz* 
Factors 
pee ee a, 

Scales I II I IV Communalities 
Hs T7177 1596 —3732 0812 7761 
D 7381 —4463 —0736 0067 7494 
Hy 7718 2705 —3441 —0800 7937 
Pd 6469 2812 1456 —1442 5395 
M-F 3326 2754 —0010 2153 2328 
Pa 6993 —1476 0602 —1917 5511 
Pt 8682 —1356 1381 0514 7938 
Se 8332 1269 2288 0448 7647 
Ma 3747 4210 2973 0357 4073 
Si 5930 —5389 0881 0912 6581 


Bi 5880 O 
* Al decimals omitted. 

greatest loss in communality is about six to seven percent asso- 
ciated with the Hy, Hs, and Ma scales; the loss in Sc, Pd, and Pt is 
about three to five percent; and the communality changes in D, 
M-F, Pa, and Si are negligible. In most cases, however, the com- 
munality loss is minimal, and, in general, the first three factors in 
Table 3 resemble very closely the three factors in Table 4. 


TABLE 4 
Factor Matriz* 
(From Corrected Correlations) 
wee NE ee 
Factors 
Ee ee Commu- 
Beales, 1 re ey v wW nitis 
Hs 7576 1501 —3218 1648 —0748 —0010 7328 
D 7267 —4616 —0414 0391 0015 —0197 7448 


Hy 7395 2703 —3133 —1483 0682 0071 7447 
Pd 6333 2987 1334 —1175 —0487 —0220 5248 
MF 3332 2850 —0026 1937 0742 —0009 2352 


Pa 6933 —1609 0031 —2026 0028 0042 5476 
Pt 8576 —1081 1092 0214 —0415 0314 7622 
Se 8186 1389 1860 “0491 0256 —0140 7273 
Ma 3795 3844 2953 0240 —0006 0096 3797 
Si 6129 —5183 1008 0638 0395 0033 6600 


S 6129-5183 1008 08 


* All decimals omitted, 


A Both sets of factors were machine-rotated by the varimax Tou- 
tine. The rotated factors in Table 5 are from the factor structure 
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in Table 3; those in Table 6, from the factor structure in Table 4. 
Again, the first three factors in Table 5 correspond very close to 
the three factors in Table 6; all other factors are again residual and 
therefore will be excluded from further analyses. 


eee 


TABLE 5 
Rotated Factor Matrix* 
n uM 
Factors 
eee ee ee mma Commu- 
Hs 3570 2590 7573 0897 7761 
D 8080 0430 3072 —0162 7494 
Hy 2721 3362 7757 —0705 7937 
Pd 2594 5766 3363 1635 5395 
M-F 0400 3429 2619 2123 2328 
Pa 5833 2827 2906 —2157 5511 
Pt 7135 4305 3147 0176 7938 
Se 5182 6310 3130 0114 7647 
Ma 0078 6284 1104 0156 4073 
Si 8060 0157 0696 0590 6581 
* All decimals omitted, 
TABLE 6 
Rotated Factor Matriz* 
(From Corrected Correlations) 
Factors 
Commu- 
Scales I Ii I Iv v VI nalities 
Hs 3681 2780 6802 1887 1473 0004 7328 
D 8218 0564 2527 0360 0275 —0204 7448 
Hy 2655 3416 7386 —0435 —1007 0000 7447 
Pd 2379 5811 3460 —1009 0040 —0234 5248 
M-F 0331 3411 2513 2338 0023 0003 2352 
Pa 5847 2480 3307 —1746 -0662 —0005 5476 
Pt 6815 4309 3287 0147 0533 0319 7622 
Se 4931 6090 3287 0712 -0033 —0140 7273 
Ma 0323 6063 1004 0289 0004 0110 3797 
A ee 
* All decimals omitted, 


Interpreting factors is always a difficult task. The personal-social 
correlates given below will provide meaningful stereotypes to the 
extent that an individual’s MMPI profile is weighted with the par- 
ticular factor and within the limits of the validity of the MMPI re- 


search on non-clinic samples of college females (Drake and 
Oetting, 1959; Black, 1953). 
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Starting with the highest weights and moving to lowest, the first 
rotated factor is determined by D, Si, Pt, and Pa. Sc has a moderate 
loading on this factor too, but has slightly higher loading on Fac- 
tor II. Young women with high loadings on this factor would 
tend to be seen by others as: shy, sensitive, sentimental, prone to 
worry, serious, natural, and with interests centered in home and 
family. Social submissiveness and emotional warmth in a serious 
prone-to-worry person summarizes the main traits. Under stress or 
in a college counseling situation, these women are likely to be in- 
secure, self-conscious, and indecisive. They feel a lack of skill with 
the opposite sex and are especially distractible in their studies and 
tense on examinations. 

The second rotated factor has major loadings on Sc, Ma, and Pd 
with Pt showing an appreciable loading also. It is of interest that 
the M-F scale has its highest loading on this factor, although the 
absolute value of its loading is not high. Black (1953) studied the 
ratings of normal college women with MMPI profiles weighted like 
this factor and found a mixture of flattering and unflattering terms 
used by their peers and themselves. Other girls saw these women 
as thoughtful, idealistic, and persevering, but also as self-centered 
and infantile, boastful and fickle, unemotional and self-dissatisfied. 
In self-descriptions, these women said they were polished, relaxed 
and thoughtful, but also said they were secretive, eccentric, 
and gloomy, and inarticulate. Compared with Factor I, this factor 
is defined by the characteristics extroversion, social assertiveness, 
independence of thought and action, and a tendency toward un- 
settled personal identification and philosophy. Under stress or mild 
breakdown conditions, these women will likely be confused, rest- 
less, verbal, and resistant in counseling or authority relationships. 
The loading on Ma alone seems to signify a relatively well social- 
ized expression of energy. To the extent that the Sc loading is 
Tepresented, the picture moves toward a more persistent and less 
Socially desirable hypomania. 

Determined almost exclusively by Hy and Hs, Factor III rep- 
resents a combination of loadings which is relatively common in 
the MMPI profiles of many normal and clinic groups. Again 
Black’s (1953) work with normal college girls suggests this factor 
Will identify women with a general impunitive attitude in the way 
they view others, the world, and themselves. Their form of ex- 
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pression is often bland and innocuous. In sharp contrast, other col- 
lege girls describe these people as selfish, self-centered, and having 
many physical complaints. They also labeled them as neurotic, de- 
pendent, indecisive, high-strung, hostile, irritable, and lacking in 
self-control. Thus, normal women with this pattern see themselves 
as cheerful, outgoing and optimistic—looking on the bright side of 
everything. Because of their repressive tendencies, their self- 
preception is often incomplete and inaccurate. Others see them as 
flighty, immature, and sometimes naively hostile people who may 
develop physical symptoms under stress. They are resistant to im- 
plications of psychological disorder. In the counseling situation, 
Drake (1959) has found these college women outgoing people, 
who are able to talk about their problems easily. They are often 
lacking in academic drive, find themselves blocking and tightening 
on examinations, and preoccupied with conflicts with their par- 
ents and home. Their lack of academic drive seems to be related to 
their rather strongly marriage-oriented view of college life. Be- 
cause of this fact, it would be interesting to compare the aca- 
demic success of this latter group with the success of students 
loading higher on the other two factors. 

As noted previously, the two rotated factor structures, except 
for the residual factors, are quite similar in terms of variables with 
high and low, or moderate, loadings. A cursory examination would 
indicate, therefore, that the random correlations do not have much 
effect on the basic factor structure of the MMPI, but the method 
of canonical correlation is a much more systematic method fo 
comparisons. ) 

Factor scores were obtained for the 168 subjects on each of the 
rotated factor structures, using the first three factors, in Tables 5 
and 6 by the formula, in matrix form 

F= ZS (1) 


where, for p variables, N persons, and k factors, F is a N X k 


matrix of factor scores, S if the p X k matrix of factor loadings, 
and Z is the N X p matrix of standardized scores. Formula (1) isa 
modified form of Harman’s (1960, p. 341) formula (16.10). 

| Harman’s formula involves the inverse of the variables’ correla- 
tion matrices and produces sets of orthogonal factor scores which 
would tend, operationally, to maximize similarities between corre- 
sponding factors in terms of the canonical correlation analysis. 


) 
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Formula (1) above will produce correlated factor scores that dif- 
fer from one set of factors to the other only in terms of S, the 
matrix of factor loadings, (For discussion of a similar point, see 
Pinnean and Newhouse, 1964, pp. 272-273.) 

The factor scores’ intercorrelations are presented in Table 7 
which, for all intents and purposes herein, contains a supermatrix 
as shown by the partitioned blocks. The upper left and lower 
right matrices in Table 7 contain respectively the correlations 


TABLE 7 
Factor Score Correlations 
U FACTORS C FACTORS 
FACTORS I Il II I Il Il 

I 1.0000 0.8475 „0.9082 0.9999 0.8546 0.9103 

U I 0.8475 1.0000 0.9350 0.8454 0.9998 0.9409 
Ill 0.9082 0.9350 1.0000 0.9071 0.9398 0.9997 

I 0.9999 0.8454 0.9071 1.0000 0.8526 0.9091 

c 0.8546 0.9998 0.9398 0.8526 1.0000 0.9452 
Il 0.9103 0.9409 0.9997 0.9091 0.9452 1.0000 


among the U factors (from Table 5) and among the C factors 


(from Table 6); the lower left and upper right matrices, being 


transposed of each other, contain the cross-correlations between the 
two sets of factors. The correlations between the corresponding 
factors I, II, and III in U and C, respectively, have correlations all 
approaching unity in value. 

The canonical correlations between the two sets of factors are 
presented in Table 8. Notwithstanding the high inter- as well as 


TABLE 8 
Canonical Correlations 
Canonical Variables 

Canonical 8S 
Variables 1 2 3 

1 0.9999 0.0004 —0.0001 

2 0.0004 0.9996 —0.0001 

3 —0.0001 —0.0001 0.9947 


intra- correlations between individual pairs of factors, three very 
high, distinct correlations appear between the two sets of data. This 
result shows that there are three linear compounds, combining the 
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factor scores in each set of factors, that will produce extremely 
high correlations between the two factor structures, which further 
substantiates the fact that the random correlations have little, or no, 
effect on the internal structure of the MMPI for this study. More- 
over, the occurrence of three extremely large, independent ca- 
nonical vectors, as indicated by the near-zero off-diagonal corre- 
lations in Table 8, lends some credence to the system being three 
dimensional in structure, but a more detailed examination is re- 
quired. 

The equations for combining the U and C factors (from Table 
7) to produce each of the canonical correlations are presented in 
Table 9; for instance, by entering each subject’s factor score ap- 


TABLE 9 
Equations for Producing Canonical Correlations 
from Individual Factor Scores 
Canonical 
Correlations EQUATIONS 
FIRST Us = 0.4292 I + 0.3419 II + 0.2670 III 
Cı = 0.4176 I + 0.3446 II + 0.2740 III 
SECOND Uz = —1.9715 I + 1.5489 II + 0.4263 III 
C: = —1,9912 I + 1.5686 II + 0.4163 III 
THIRD Us = —1.2791 I — 2.3330 II + 3.5417 III 


C: = —1.2796 I — 2.6137 IL + 3.8159 III 


propriately into the equations, U, and C, will correlate at .9999, 
Uz and Co, at 9996, and Us and Cs at .9947. The nearly identical 
weights for factors I, II, and III in U and C respectively occurs 
because the pairs of factors are each so highly related as to be 
co-linear, for all intents and purposes. The order of relative im- 
portance of the factors in determining the relationship between the 
two structures in the first canonical correlation, as indicated by 
their weights, is I, II, and III. The order is the same in the second 
canonical correlation, except that Factor I has an opposite (nega- 
tive) sign. In the third canonical correlation the order is reversed, 
except that here the third factor has the opposite sign. 
Notwithstanding the reluctance of previous researchers (e.g. 
Wheeler, Little, and Lehner, 1951) to account for scoring artifacts 
in the MMPI scales, it would appear from the results of the pres- 
ent study that such artifacts do not have a great deal of effect on 
the internal structure of the battery for the ten basic clinical scales. 
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If more scales are developed from the same set of items, however, 
the amount of item overlap might indeed become a problem and 
consequently, affect the internal factorial structure, such as in the 
study of Kassebaum, Couch, and Slater (1959). The method of 
canonical correlations, in any case, is a useful method for com- 
paring resulting structures. 
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In December 1963, the 80,842 candidates electing the English 
Composition Test (ECT) in the College Board test administration 
were required to write a 20-minute essay. The test was the first 
ECT to include free writing since December 1956 and the first to 
include a 20-minute essay since April 1947 (CEEB, 1947). In a 
period of five days during the subsequent week, 145 readers gave 
each of the essays two independent readings. This study was de- 
signed to determine whether under these conditions the reading 
reliability would approach that obtained previously under some- 
what less demanding circumstances. It culminates a 15-year period 
of research and development during which the College Board 
English Composition Test has moved from an all-essay examina- 
tion through various stages of objective and semi-objective format 
to its present specifications. 

The decision to reintroduce a 20-minute essay had not been 
made arbitrarily. Using data collected in 1961, Swineford (Noyes, 
1963) had demonstrated that 20-minute essay scores based on five 
independent readings contributed uniquely to the validity of a one- 
hour English Composition Test for predicting a reliable essay 
criterion. The reading had been carried out during a five-day ex- 


1 We would like to thank Ledyard R Tucker for his immeasurable aid in 
the analysis of this experiment, Paul B. Diederich, Frances Swineford, and 
Karl G. Jéreskog for their comments on the manuscript, and Lynn K. Gaines 
for her assistance in gathering the data. 
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perimental reading by 25 carefully selected CEEB readers, and an 
analysis of the time required had indicated that as many as three 
readings might be obtained at a cost equivalent to that for reading 
the semi-objective interlinear exercises used periodically since 1951. 
The following year Godshalk, Swineford and Coffman (1965) dem- 
onstrated that Swineford’s original findings could be duplicated 
during a half-day reading session by the 146 readers assembled to 
read the December 1962 interlinear exercise. They also found that 
substantially all of the unique validity of the 20-minute essay could 
be obtained from only two or three readings instead of four or 
five, particularly if a four-point scale were substituted for the 
original three-point scale. 

The basic question asked in the current study was whether or 
not reader reliability would hold up over the five-day period re- 
quired for reading over 80,000 essays. If so, presumably the valid- 
ity findings of the experimental studies could be expected to apply. 
A related question of interest concerned the extent to which 
readers actually did follow the instructions to give a “single global 
judgment” rather than to respond to details of an essay. Subse- 
quent sections of this report present the data which have led us to 
conclude that the operational reading was as reliable as previous 
experimental readings and that the readers were giving global 
judgments, 


Method 


Readers 


Of the 145 readers, two were “chief readers” and 18 were desig- 
nated as “table leaders.” The chief readers were responsible for the 
conduct of the reading in general. The table leaders took care of 
any minor problems that arose or questions from the readers at 
their tables. Their main function, however, was to “spot-check” 
their readers and warn them when they seem to be grading con- 
sistently too high or too low. The remaining 125 readers read and 
graded the essays. The readers represented both public and private 
high schools and colleges. There were 42 women and 103 men. 


Instructions 


The general instructions to the readers were to read the essays 
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“holistically” and then assign a grade, 1 to 4. The interpretation of 
these grades was as follows: 
. obviously below a reasonable standard 
2. not sufficient promise or competence to be considered in the 
upper half 
3. clearly competent, promise of effective performance 
4. superior; not perfect but very good; effective. 

The essay was to be graded on its impact. Individual petty an- 
noyances or pet peeves (e.g., spelling errors, split infinitives, etc.) 
were to be ignored as much as possible. Each table had its own 
letter code for the grades assigned, so that a second reader from a 
different table would not know the grade assigned by the previous 
reader. 


= 


Experimental Design 


The experimental design called for a “control” group of papers 
to be read each day by a specific number of readers. Twenty-five 
papers were selected from the incoming essays representing as 
wide a geographical distribution as possible. The essays were hand- 
reproduced so that the end-product was indistinguishable from the 
originals. In this way the control papers could be presented to the 
readers along with the other essays without detection. A random 
selection of 25 judges per day was made prior to the conduct of 
the experiment. Hach reader was presented a set of the 25 control 
papers, mixed with the other essays he was to read. Unforeseen 
circumstances (e.g., some judges’ failure to read all of the control 
set of 25 papers on the day they were presented for reading) 
caused a variation in Ns from 18 to 25 over the five-day period. The 
readers graded the controls just as they did the others, on a four- 
point scale, letter-coded their grade, entered their reader’s number, 
and went on to the next essay. 


Results 


Reliability of the Judgments 


The means and standard deviations of the 25 papers are shown 
for each day in Table 1. The variation of means was well within 
that expected by chance. An analysis of variance procedure 
(Winer, 1962) was used to assess the reliability of the judgments 
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TABLE 1 
Means and Standard Deviations for the Five Reading Days 


made by the subjects. This procedure provides an estimate of the 
average correlation between judges by comparing the variance be- 
tween papers with the variance within papers. 


TABLE 2 
Summary Analyses of Variance for Each Day of the Reading 


Day Source of Variation ss df MS F re n 
1 Between papers 182.34 24 7.59 21.08 .466 23 
Within papers 199.74 550 -36 
2 Between papers 119.99 24 5.00 11.90 .364 19 
Within papers 186.84 450 42 
3 Between papers 170.48 24 7.10 25.36 .493 25 
Within papers 
4 Between papers 165.18 24 6.88 23.72 .476 25 
Within papers 173.44 600 -29 
5 Between papers 86.78 24 3.62 9.28 .264 23 


Within papers 216.70 550 +39 


= These reliabiity coefficients have been sipped | down by use of the Spearman-Brown formula. 
They represent the reliability of a single judgment 


Separate analyses were made on the data for each day. Table 2 
provides a summary of the five analyses of variance which were 
used to determine the reliability of the readings for each day. The 
first step in each analysis was to calculate the reliability based upon 
the judgments of all the readers. By use of the Spearman-Brown 
formula these reliabilities were stepped down in order to find the 
reliability of single judgment. This single judgment coefficient rep- 
resents the average reliability among all readers and across all pa- 
pers. It can be seen that these reliabilities range from a high of .493 
on day three to a low of .264 on day five. These coefficients, of 
course, represent the degree to which the readers could agree upon 
the quality of each paper; mean differences across days, of course, 
lower the overall reliability. 
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There is not, unfortunately, any well-developed procedure for 
determining the extent to which the variations among these five 
reliabilities can be attributed to chance alone. An appropriate test 
would require the ability to make multiple comparisons among F 
ratios. It is possible to evaluate the difference between two F ratios 
through a procedure presented by Schumann and Bradley (1957). 

The F ratios shown in Table 2 were compared using the pro- 
cedure and the tables presented by Schumann and Bradley. The 
reliability on day five was found to be significantly different from 
the reliabilities on days one, three, and four (p < .05).? 

The .364 reliability found on day two was not significantly dif- 
ferent from any of the other four reliabilities in the experiment. 
For this reason it is very difficult to determine whether the low re- 
liability found on day five represents a significant change due to 
psychological factors or whether it, along with the reliability 
found on day two, was simply a low estimate of the population re- 
liability. Given the large number of judgments that went into the 
estimation of these reliabilities, the most parsimonious conclusion 
would seem to be that day five does in fact represent a significant 
change from the reliabilities found in the other four days and that 
day two represents a low estimate of that four-day reliability. 


TABLE 3 
Reliabilities for Single and Multiple Readers 


Coffman and Swineford 


Reliabilities Reliabilities for 
a Four Readers 
Single Two Three Four _— 
Day Reader Readers Readers Readers Topic A Topic B 
1 466 635 +723 Aref 
2 364 533 -631 695 
3 493 -660 744 +795 
4 -476 644 731 784 
5 -264 417 -518 589 
Entire 
Reading -406 577 672 -732 71 67 


2 Table presented by Schumann and Bradley is quite incomplete in that it 
does not provide for any level of significance other than .05. In addition, it 
does not provide for F ratios that have more than 30 degrees of freedom. It 
was necessary in the present analysis, therefore, to interpolate well beyond 
those values given in the table. Consequently, the only significance level which 
bi = used in those analyses involving the Schumann and Bradley procedure 

e .05. 
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Table 3 shows how these reliabilities can be stepped up with 
multiple readings. If two readers read each paper, the reliability 
jumps from .406 to .577. Since five days were required to obtain. 
two readings, perhaps special attention should be paid to the two 
reader reliability. The four-reader reliabilities were about the same 


as those found by Godshalk, Swineford and Coffman (1965) (also ` 


shown in Table 3) and demonstrated that the extended reading 
period did not lower the reliabilities. With the exception of day five, 
in fact, the present coefficients tended to be higher. 


Factor Analysis 


Even though the readers in this study have been instructed to 
make a single global judgment, there was, of course, no guarantee 
that they were not operating on different bases. In their study of 
essay gradings Diederich, French, and Carlton (1961) had con- 
cluded that the readers had made their global judgments on the 
basis of diverse “schools of thought.” It is obvious that an overall 
reliability coefficient would be reduced if the readers in a sample 
were using different points of view in making their judgments. 

A factor analysis of the covariances among the 25 papers was 
done to assess the possibility that the 118 readers in the present 
study have a small number of identifiable points of view (similar 
to Tucker and Coffman, 1959). Four factors were extracted from the 
25 X 25 matrix and a varimax rotation performed. The results of 
this analysis are shown in Table 4. The reader should note that the 
communalities and loadings are not based on unit variances. Rather, 
each paper has a variance which is related to the mean rating re- 
ceived by the paper. Since these values are less than unity, the 
communalities appear small for those accustomed to using com- 
munalities in factor analyses of correlation matrices. The unro- 
tated data are given in the appendix. 

With a few exceptions, quite good simple structure was achieved 
with the varimax rotation. That is, a paper was loaded fairly high 
on one factor and reasonably low on all the others. When the papers 
were grouped according to the factors they seemed to represent, it 
was noted that there were great similarities in the average grade re- 
ceived by those papers. In Table 5 it can be seen that the poor pa- 
pers tended to load on factor two while the good papers tended to 
load on factor three. 


b 
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TABLE 4 


Varimax Factors 
———E nil ivi a_i 


sD 
Paper Communalities of Papers Factori Factor2 Factor3 Factor 4 
1 18 61 —01 33 08 27 
2 12 59 00 08 12 31 
3 07 62 —01 13 24 04 
4 13 68 —03 18 —01 31 
5 08 52 02 27 09 —04 
6 13 54 —01 15 26 20 
7 12 60 14 —02 25 18 
8 11 57 06 10 30 —04 
9 13 54 08 08 —01 34 
10 30 68 10 —04 54 00 
11 11 52 04 27 13 14 
12 10 48 06 29 08 06 
13 06 61 07 —05 21 09 
14 14 60 08 34 04 14 
15 14 66 21 06 12 27 
16 11 58 16 25 —01 15 
17 13 54 11 31 —05 13 
18 06 48 23 10 03 01 
19 15 55 38 —03 09 05 
20 22 59 44 12 —01 11 
21 24 65 36 33 —06 —01 
22 19 56 22 36 -11 04 
23 11 59 25 00 07 21 
24 22 67 44 11 11 04 
25 12 51 32 07 09 —02 


TEAVE NEU NESE STEN E 


The temptation to start naming factors at this point was success- 
fully resisted. Instead, papers were grouped according to the fac- 
tors they loaded on. Analyses of variance were computed for the 
group of papers identified with each factor for each day. In other 
words, the same type of analysis as was done earlier was repeated 
for the four factors. Two papers, Numbers 21 and 22, were not 
included in these analyses because they had such high loadings on 
two of the factors. Since there were four factors and five days this 
meant, of course, that 20 such analyses were performed. 

The immediate interest in these analyses was to investigate the 
discriminability that existed between papers that loaded on the 
same factor. The median F ratio (out of the total of five, one for 
each day) is shown for each factor in Table 6. Although all these 
values were significant at at least the .05 level, there were consider- 
able differences in the degree to which the clusters of papers were 
discriminable from each other. The ws which indicate the percent 


48 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


TABLE 5 
Means of Papers on the Various Factors 


Range of 
Factor Paper Mean Score Mean Scores 


I 18 1,96 


= 
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TABLE 6 
Analyses of Variance to Assess Homogeneity of Means of 
Papers on Each Factor 
Factor Source of Variation 8S df MS F w? 
1 Between Judges 15.10 22 .69 2.52 05 
Between Papers 2.06 5 41 
Residual 17.94 110 -16 
Total 35.10 137 
2 Between Judges 16.66 18 -92 3.66 -10 
Between Papers 4.30 6 72 
Residual 21.13 108 -20 
Total 42.09 132 
3 Between Judges 8.49 18 .47 18.90 42 
Between Papers 23.13 5 4.63 
Residual 22.04 90 +24 
Total 53.66 113 
4 Between Judges 16.54 24 .69 6.61 sil 
Between Papers 4.59 3 1.53 
Residual 16.66 72 .23 


Total 37.79 99 ve 
a IT i o 


+ 
K 
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of variance which was determined by the discriminability between 
papers (Hays, 1963) were quite different for the four factors. 
Factor three, which included all of the good papers, had 42 percent 
of the variance of the judgments determined by the discriminabil- 
ity between papers. The next largest o? was .11 for factor four. As 
far as significance testing is concerned, the F ratios for papers de- 
fining factor three were significantly larger than the ratios for any 
of the other factors and the F ratio for factor four was significantly 
larger than that for factor one. 
Discussion 

When the factor analysis had been completed, the papers were 
grouped by factors and read by the authors and others. There cer- 
tainly did not seem to be any obvious reason why the papers were 
grouped this way, except for their general quality. Poor papers 
tended to be poor on every attribute of composition and good 
papers tended to be good on these attributes. Students who have 
difficulty organizing papers also have difficulty in spelling, ete. It 
seemed that the factors did not represent content dimensions and 
that some other type of explanation for the factor structure would 
be necessary. 

In contrast to what is usually done in a factor analytic study, no 
attempt will be made here to provide labels which will identify the 
factors. On the contrary, we will present an argument to the effect 
that these factors do not represent different sources of content. We 
will argue that they represent diverse points of view in the relating 
of grades to the quality of a paper. 

Let us consider how we might have interpreted the data if only 
factors 2 and 3 had emerged. That is, suppose all the good papers 
were clustered together and all the poor papers were clustered to- 
gether in a two-factor space. It would not matter in this situation 
how any individual assigned grades, just so long as he tended to 
give all the good papers the same type of score and to give all the 
bad papers the same type of score. Thus, some readers might give a 
score of 1 to the poorer half of the papers; others might assign a 2 
to this half; and there might even be a few who give a 3 to all of 
the poorer papers. Similarly, they would assign the same score to 
the better half (2, 3, or 4). Suppose further that all the readers had 
perfect agreement as to which were the good papers and which the 
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bad. Figure 1 shows the hypothetical function that would describe 
scoring of this type. In this situation there is no discriminability 
among papers in the good group nor is there any discriminability 
among papers in the poor group. A factor analysis of these data 
would show that there were two factors, and those factors would 
undoubtedly be identified as good papers and poor papers. 


4 
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Good 
Quality of Paper 

Figure 1. Hypothetical relationship between quality of paper and grade as- 

signments with no differences in “eg aap among papers at a given 

Similarly, if there were three categories of papers instead of two, 
and if all the judges continued to be equally discriminating within 
the three categories, we would find that the data would produce 
three factors. 

A fourth factor could be introduced if there were differences 
among the readers with respect to the location of the range be- 
tween the two cutoff points. Thus some papers that were in the 
middle range for some readers would be in either the upper or 
lower categories for other readers, not because they valued the 
papers differently, but because they simply used different cutoff 
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points for classification. The shifting of the middle range up and 
down would cause certain papers to correlate which were quite 
disparate in mean score while they maintained correlations with 
papers with similar scores. This is precisely what occurred with 
those papers which loaded on factor 4. 

It should also be noted that there were differences in the dis- 
criminability among the papers found on a factor. There was a 
good deal of discriminability among the better papers and very 
little among the poorer ones. This indicates that the function relat- 
ing the quality of the paper to the grade assignment was curvi- 
linear. 


A 


a 
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Fae Quality of Paper toog 
Figure 2, Hypothetical relationship between quality of paper and grade assign- 
ments with variable discrimination. 

a The reader will note that the range incorporated here represents the dis- 
crepancy between the most lenient and the most severe reader. The most 
lenient reader includes many papers in the highest category while the most 
severe reader includes few; the opposite is true with papers in the low cate- 
gory. 


Figure 2 depicts a functional relationship between grades and 
quality which is very descriptive of the data found in this study. 
The breaks at the cutoff points are intended to communicate that 
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there is a cutoff range which is the composite of all the readers 
used in the study. It can be seen that there is differential discrim- 
inability among papers in various parts of the range. Data of this 
sort would produce four common factors. 

Explaining the factor structure in this manner does not imply 
that there is not a variety of attributes used in the judgment. The 
judges, to be sure, must have been sensitive to differences in gram- 
mar, spelling, general style, etc. These are the attributes which the 
judges used as criteria to make their assignment to the categories 
good, bad, and mediocre. Why then, we might ask, aren’t these at- 
tributes represented by the factors instead of the function relating 
quality and grade assignment? Why have we chosen to represent 
these data as if they described a simplex? 

Guttman (1954) has described a simplex as a structural relation- 
ship that exists between elements when these elements are ordered 
on a single dimension of complexity. Complexity is used in this 
context in a most general way; a complex element is made up of 
many factors while a less complex element is made up of fewer fac- 
tors. In the present setting, essays which have good spelling, good 
grammar, good ideas, good style are complex in the sense that they 
manifest the existence of many positive attributes. A poor paper, on 
the other hand, is “simple” in that none of these attributes may be 
present. Thus, the ordering of papers by their mean could be con- 
strued as an ordering by complexity. It has been pointed out al- 
ready, of course, that with the present data an ordering by factors 
is almost equivalent to ordering by means. 

A description of the data in these terms is most satisfying in that 
it suggests that the readers were making global judgments with re- 
spect to quality using a set of interrelated criteria. This is consistent 
with their instructions and the fact that they took about a minute 
to read each paper. It also suggests, in contrast to the Diederich, 
French, and Carlton study, that, although all the readers may have 
assigned different weights to the various attributes of a composition 
(e.g, spelling, grammar, organization), each of the individual at- 
tributes tended to be given the same weight by the readers. That is, 
it does not seem as though some judges rated the papers primarily 
upon grammar, while other judges rated the papers primarily upon 
style. Rather, this explanation suggests that the relative weighting 
between grammar and style is roughly the same for all judges. 


m 


a 
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The Problem of the Fifth Day 


The reliabilities found in the present study were slightly higher 
than those found in previous investigations of the short essay. The 
severe drop on the fifth day, however, does pose certain practical 
problems, Although it is quite understandable that the readers 
should suffer a loss in vigilance as they neared the end of this fairly 
grueling task, our understanding, per se, does very little to combat 
the problem. The drop in reliability to .26 on the last day poses a 
serious problem. The immediate question becomes, therefore, Is it 
possible to maintain a high state of vigilance at the end of the read- 
ing period? If the readers are mature and conscientious people, as 
they were in the present setting, they might resolve the problem 
themselves if it were pointed out to them that there was a tendency 
for them to slip at the end of the reading period. If, however, the 
readers were not able to maintain a state of vigilance by their own 
efforts, then the reliability problem will appear more troublesome. 

We have been assuming that the drop in reliability occurs at the 
end of a reading period because the readers are anticipating the 
completion of their task, If this is so, it means that there would be 
an equivalent drop in reliability regardless of how long the reading 
period was. This implies immediately, of course, that this problem 
cannot be handled by simply shortening the reading period by 
any small amount. It would seem that some external source would 
be needed to bolster the reader morale and effort. Future research 
efforts should be concerned with experimental evaluation of such 
external efforts. 
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Ir college selection is to be based in part upon achievement test 
scores, it is important that such scores have these characteristics; 

1. The achievement test scores of students should be increased 
by taking the course of study for which the test has been devel- 
oped. 

2. At least some of the increase should be relatively permanent. 

8. Any procedure that affects achievement test scores should 
similarly affect the college criterion measure. 

One set of procedures for which there is some question regard- 
ing all three points is that of “coaching” practices. The every- 
day usage of the term “coaching” generally refers to intensive 
teaching, of either individuals or groups of students, that takes 
place outside of the regular school program of instruction, and that 
is specifically directed toward obtaining high test scores, rather 
than toward a good education. 

The present study provides information regarding the effects of 
coaching on the College Board English Composition Test (ECT). 


Method 
Design 
The basic experimental paradigm was to contrast performance 
on the English Composition Test of a group of students given 
coaching with that of an otherwise comparable group of students 
not given coaching. Students were randomly assigned to the two 


groups. Scores on the Verbal section of the Preliminary Scholastic 
Aptitude Test (PSAT-V) were already available for the students, 
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"and served as an additional check on the initial comparability of 
the groups. The paradigm, then, was that of a Pretest-Posttest 
Control Group Design (Campbell and Stanley, 1963). This basic 
paradigm was complicated in the following ways: 

1. To assess how possible effects of coaching-might vary with 
the situation, comparable coached and non-coached (control) 
groups were set up within each of six different secondary schools. 
In this way the composite effects of differences among the 
schools in procedures, personnel, and students could be ex- 
amined in a nonanalytic way, that is, without isolating specific 
sources of variability between the schools in coaching effects. 

2. One might reasonably expect within a given school “leakage” 
of coaching, that is, the coached students passing on the benefits 
of their extra instruction to their non-coached fellow students. 
Such leakage would reduce the estimated effects of the coaching. 
To guard against this possibility, each of the six schools contain- 
ing both coached and control students was matched with another 
“control-only” school, the matching based upon type of school 
(public, parochial; city, suburban), and mean SAT scores of the 
schools’ College Board candidates in a preceding year (January 
and March, 1958). The pairing of coached-and-non-coached 
schools with control-only schools, then constituted a Nonequiva- 
lent Control Group Design (Campbell and Stanley, 1963). 

3. One might also reasonably expect that certain item types on 
the ECT would be more coachable than others. For this reason 
an attempt was made to estimate the relative coachability of the 
three item types that appeared on a given form of the ECT. Each 
coaching group received coaching on only two of the three item 
types upon which they were later to be tested, Item types were 
randomly assigned to schools, with the restriction that each of the 
three possible pairs of item types be assigned to two of the six 
schools. In this way the interaction of school with item type was 
deliberately confounded (Lindquist, 1953). 

4. To estimate the permanence of possible coaching effects, 
students were given a second ECT ten months after the first. 


Materials 


The ECT given in May 1961 was used as the basic criterion 
measure. It contained three item types: sentence correction (SC), 
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construction shift (CS), and paragraph organization (PO). The 
ECT given in March 1962 served as the second criterion measure. 
Tt also contained sentence correction and paragraph organization 
item types, as well as error recognition (ER) items. Examples of all 
these item types are given in Jacobs (1964). i 

The coaching materials provided to the cooperating schools 
were actual items taken from retired (no longer secure) forms of 
the ECT. The construction shift items are publicly available 
(Thomas, 1956). A total of 45 sentence correction, 16 construc- 
tion shift, and 16 paragraph organization paragraphs (comprising 
103 separate paragraph organization items) were used as coaching 
materials, 


Procedure 


Selection of Schools. It was desired to work with schools within 
the northeast metropolitan area, where the problem created for 
students, parents, and school officials by the existence of coaching 
schools is probably greatest. From a list of schools in the New 
York City-New Jersey-Philadelphia area that had 100 or more 
Scholastic Aptitude Test (SAT) candidates in January and March 
1958 and that gave the Preliminary Scholastic Aptitude Test to 
its juniors, six pairs of schools were selected. Each pair was roughly 
matched on mean SAT Verbal scores of their 1958 candidates. 
A letter was sent to the principal of each of these schools, 
explaining the purpose and general design of the study. He was 
asked to consent to cooperate without being told first whether his 
school would be selected to have both coached and control groups 
or just a control group. It was felt that if the principal were given 
this information before he agreed to cooperate, there might be a 
lower rate of cooperation among the control-only schools ap- 
proached, which might in turn produce some systematic initial 
differences between the coached-and-control schools and the con- 
trol-only schools. It turned out that when the letters were fol- 
lowed up with telephone calls, all the principals were willing to co- 
operate, and expressed mild to intense enthusiasm. In the making of 
further arrangements with the schools, one school was discovered 
not to be able to cooperate because a free classroom for coach- 
ing was unavailable during the school day lengthened by multiple 
sessions, and a thirteenth school was contacted and accepted. 
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A brief description of the cooperating schools is given in Table 1. 


TABLE 1 
A Description of Schools Cooperating in the Study 


Mean Mean 
SAT-V of SAT-Mof 
Candidates Candidates Item Types 


Code Type and in Earlier in Earlier Coached 
Name Location Year Year P.O. CS. S.C. 

Ai Public H.S., 

Long Island, N. Y. 501 516 x — X 
A: Public H.S., 

Long Island, N. Y. 501 529 — sa as 
Bı Public H.S., 

New Jersey 467 495 x x x 
B, Public H.S., ; 

New Jersey 479 507 — is = 
Cı Public HS, 

New York City 486 502 x x = 
C: Public H.S., 

New York City 486 495 —- - — 
Dı Catholic H.S., 

New York City 507 536 — X x 
D: Catholic H.S., 

Philadelphia 508 548 ary sy, 
E, Public H.S., 

Westchester, N. Y. 531 532 X x pe 
E: Public H.S., 

Westchester, N. Y. 532 544 = —- — 
Fı Public H.S., 

New York City 444 469 X Xx 
Fi Public HS, i 

New York City 445 478 ias oS ae 


PE U Alaha ia r EN AERAN Sar HO W T Cites Sete 
Selection of Coaches 


In each cooperating coached-and-control school the chairman 
of the English Department selected a member of his depart- 


ment to serve as the coaching instructor, or coach. Each coach was 
given an honorarium of $5.00 an hour. 


Selection of Students 


Within each cooperating school an announcement was distrib- 
uted to each junior thought by the school to be a likely college ‘ 
applicant. The student was asked to volunteer for the experiment, 
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knowing that through random selection he might not be selected 
at all for the experiment, and if selected, he might be placed in the 
control rather than the coached group. The student was further 
told that those selected for either the coached or control groups 
could take the ECT that year without paying the $6.00 fee, 
that they might be asked to take the ECT again the following 
year, also without fee, and that for those selected for the coached 
group there would be no charge for the coaching. 

It was intended to select randomly from among the volunteers 
in each control-only school 30 students, and in each coached-and- 
control school, 60 students, to be further randomly assigned to the 
coached and control groups. In one coached-and-control school, 
however, an administrative mix-up resulted in there being alto- 
gether only 24 volunteers. All 24 were therefore accepted into the 
experiment, and 12 of them were selected at random for the 
coached group. 


The Coaching 


Each coach was presented with coaching materials for two item 
types. He was told that these were item types that had appeared 
on the ECT in the past and were likely to occur again. The coach 
was encouraged to supplement these materials with whatever addi- 
tional material seemed appropriate, to give homework assignments, 
and to keep a log describing the coaching activity. 

From these logs it appeared that in each school roughly one- 
half of the coaching time was devoted to direct work with the 
coaching materials supplied, while the rest of the time was de- 
voted to English composition work not as specifically aimed at 
these item types. The coach used a wide range of supplementary 
material, including “cram” books, the booklet A Description of the 
College Board Achievement Tests, and Strunk and White (1959). 

The coaching was given three hours a week for six weeks. In 
some schools student motivation lagged, resulting in spotty attend- 
ance at the coaching sessions. In School Bı, on the other hand, 
student interest and speed of working was such that the coach re- 
quested additional coaching materials from ETS, and was given the 
exercises dealing with the third item type. In another school 
(School C,) both coached and control students were routinely 
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given a “Composition Conference Period” that presumably over- 
lapped in function with the coaching sessions. 

After the sixth week of coaching, all students in the study 
took the ECT. In the spring following the first ECT, the students 
in the coached-and-control schools, now seniors, were again con- 
tacted to take the ECT. As will be seen, the results available at 
that time suggested it would be pointless to retest the students 
in the control-only schools, 


Results and Discussion 


The sample of schools, located as they were in the New York 
City-New Jersey-Philadelphia metropolitan complex, and having a 
large number of College Board candidates, were well chosen for 
the purpose of the study. One might argue, however, that the 
students within these schools who volunteered to take part in the 
experiment as juniors were not part of the usual College Board 
candidate population, and so the results obtained with them might 
not be generalizable to this population. 

If the “right” students volunteered to take the ECT free of 
charge, we would expect these same students to pay for and take 
the SAT as seniors. A check of the ETS records was made, and the 
results are presented in Table 2. This supports the assumption that 
the students volunteering to take part in the experiment were mem- 
bers of the usual College Board candidate population. 


TABLE 2 


Number of Students in Experiment Taking SAT between 
December 1961 and August 1962 in Each 


Coaching-and-Control School 
Number of 
School Students 

Ai 54 

Bı 54 

Cı 44 

Dı 57 

Ei 22* 

Fi 35 


* In each of these schools except E1 the total i k 
in En it was 24. pt Ey number of students in the experiment was 60 


The next analysis was to estimate the possible effects of leak- 
age of coaching from coached to control students within the 
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same school. Leakage would be demonstrated by the control 
group in a coached-and-control school doing better than the con- 
trol group matched with it in a control-only school. 

Control group means of ECT scores in the two types of schools 
are shown in Table 3. The differences in means were relatively 
small, and not consistent in direction. We conclude that they reflect 
imperfect matching of schools and error of measurement. Since 
leakage appeared to be negligible, the students in the control-only 
schools were not asked to retake the ECT as seniors. It was assumed 
that any leakage would already have occurred by the time the 
first ECT was taken. 


TABLE 3 
Comparison of Control Group Means on ECT 
in Each Type of School 
Coaching and Control School 

School Control School Only 

A 475 498 

B 495 487 

Cc 457 455 

D 482 496 

E 554 551 

F 418 422 


It was appropriate, therefore, to compare the coached with the 
control students within each coached-and-control school to esti- 
mate the possible effects of coaching. An initial analysis made was 
of total test score on the three item types of the ECT. The results 
are presented in Table 4. The scores are given in terms of the Col- 
lege Board: scaled score (mean = 500, ¢ = 100). In two of the 
six coached-and-control schools, there was essentially no difference 
between coached and control groups, while in each of the other 
four schools, the mean difference of from 44 to 73 points favor- 
ing the coached group was statistically significant and also presum- 
ably practically significant. We have, then, rather clear evidence 
of over-all coaching effects in some schools, and not in others. 

The next analysis was aimed at exploring the mechanism by which 
the coached groups achieved their superiority. Did they get 
fewer items wrong and/or omit fewer items, and was this confined 
to those item types for which coaching was received? Two statis- 
tically independent part scores were derived from each item type 
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TABLE 4 
Total Test Score Results on ECT; 

PSAT-V-Mean ECT; Mean Analysis of Covariance 
Coached Control Coached Control Adjusted 

School Group Group Group Group df M.S. F 

Ay 47.8 46.5 519.3 474.8 Between 1 16378.1 4,24* 
(29)*** (25) Within 51 3864.9 

B, 43.8 43.1 540.9 495.1 Between 1 20444.0 4.70* 
(24) (27) Within 48 4348.1 

C; 47.4 44.1 456.4 457.2 Between 1 3510.6 — 
(11) (24) Within 82 5454.9 

D: 44.4 46.1 475.4 481.8 Between 1 526.7 _ 
(25) (22) Within 44 3532.9 

E, 55.2 55.6 627.0 554.3 Between 1 33601.0 10.75** 
(12) (12) Within 21 3126.5 


F, 44.0 41.3 480.0 418.3 Between 1 20013.3 6.96* 


(26) (23) Within 46 2877.5 
*p <05 


*p <0 
*** Number of students, 


for each group: the number of items answered incorrectly, and the 
number of items not attempted. The results are shown in Table 5. 

Ignoring item types, it appears that in each school showing an 
over-all coaching effect, the coached group achieved its superior- 
ity by answering fewer items incorrectly; regarding omits, there 


noted previously (Frenċh, 1955). It is interesting that in School 
Cı, which does not show an over-all coaching effect, the coached 
group shows the same pattern of fewer wrongs as seen in those 
schools that do show an over-all coaching effect, and shows al- 
most double the number of omits, 

Does this pattern occur only on those item types for which 
there was coaching? This seems to be true only regarding number 
wrongs for School F}. 


The general Conclusions, then, from this inquiry into the mech- 
anism of coaching effects, are: 


1. Coached groups achieve their superiority by getting fewer 
items wrong. 


2. In general this effect is not limited to those item types coached 
for. 


a a 
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TABLE 5 
Number Wrongs and Number Omits by Item Type for Each Group 


Item Wrongs Omits 
School Type Coached Control Coached Control 
Ai 8.C. 11.3 14.1 5.2 3.4 
P.O. 12.1 15.3 4,2 4.1 
C.8.* 9.0 12.0 6.8 5.9 
Total 32.4 41.4 16.2 13.4 
Bı 8.C. 8.4 13.0 6.2 5.6 
P.O. 11.6 12.6 4.2 4.6 
Cs. 7.5 8.4 8.6 8.9 
Total 27.5 34.0 19.0 19.1 
CQ 8.C.* 12.4 15.0 8.9 3.4 
P.O. 12.2 18.5 9.0 2.2 
Cs. 9.8 11.2 8.4 TT 
Total 34.4 44.7 26.3 13.3 
Di 8.C. 12.7 13.3 4.8 4.8 
POS 15.0 14.3 6.0 4.5 
Cs. 10.0 11.2 6.4 6.4 
Total 37.7 38.8 17.2 15.7 
E: 8.C.* 9.4 11.5 1.4 3.2 
P.O. 7.8 11.1 0.5 1:7. 
Cs. 6.5 10.5 4.5 3.8, 
Total 23.7 33.1 6.4 8.7 
F, S.C. 14.6 16.3 5.8 6.5 
P.O. 14.4 16.3 4.2 6.2 
CBF 12.5 11.8 6.2 7.0 
Total 41.5 44.4 16.2 19.7 


— Ow ae el eee 
* Not coached for at this school, 
3. In one school not showing an over-all coaching effect there is 


evidence from the part score analyses of some effects of 
coaching. 


Permanence of Coaching Effects 
How permanent are the gains achieved by the coached group? 
A comparison of over-all scores on the retest of the ECT after ten 


months is shown in Table 6. 
In Schools By, Dı, Ex, and F;, both coached and control groups 


appeared to have gained as a result of practice and growth. The 
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TABLE 6 
Total Test Score Results on ECT: 
lesen VAIBA OT eee ee 
PSAT-V Mean ECT: Mean Analysis of Covariance 
Coached Control Coached Control Adjusted 
School Group Group Group Group df MS. F 
Ai 46.4 52.0 519.8 569.6 
(9)* (5) 
Bı 43.9 44.8 570.8 550.8 Between 1 9258.0 2.32 
(23) (21) Within 41 3987.5 
C, 41.5 32.5 428.5 437.5 
(2) (2) 
Di 47.8 45.7 536.2 509.5 Between 1 159.4 — 
(7) (15) Within 29 35656 — 
E, 54.3 56.1 639.7 647.4 Between 1 539.7 ome 
1) (8) Within 16 5491.8 
Fı 49.5 42.8 557.1 491.7 Between 1 455.2 — 


(11) (12) Within 20 7297.5 


covariance analyses, however, indicate that the gain for the con- 
trol groups is large enough for them to catch up with the coached 
groups. Such differences between coached and control groups in 
mean score as are observed on ECTs are accountable for in 
terms of initial differences in PSAT-V scores, In Schools A, and 
Cj the small number of cases do not permit any conclusions to 
be drawn. Although the pattern of results for each school differs 
rather widely, an overview of results on PSAT, ECT, and ECT: 
for all schools combined is shown in Figure 1. p 

In the present study intensive instruction involving in part the 
use of actual items from former examinations with motivated stu- 
dents from the College Board candidate population yielded an in- 
crement in scores relative to a noncoached control group. This in- 
crement had disappeared by ten months as measured by retest, and 
presumably would not affect the college criterion measure. 

Does, then, the candidate who receives coaching have an ad- 
vantage over the candidate who does not? The present results in- 
dicate that he may indeed, but the considerable variability from 
school to school in coaching effects suggests that this will depend 
on some specifics of the coaching situation. Perhaps any analysis 
of the social importance of coaching effects on an achievement 
test should compare these effects with the effects of ordinary teach- 
ing. The test-retest data of the control groups show an interest- 
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coached. 
controls in Coaching School 
—————-=— controls in Control Schools 


a (times zP ET ETa 
Initial test (after coaching) (10 months later) 


Figure 1. An overview of test results for all schools combined. 


ing parallel here: ECT. minus ECT;, which may be taken as an 
estimate for those students taking both tests of the combined 
effects of practice and of ten months’ growth, varied from 15 points 
at School D, to 94 points at School E,. Apparently the advantage 
accruing from an additional year’s study of English will also de- 
pend on some specifics of the teaching situation. It may be, then, 
that an appropriate program of self-study by the individual would 
provide an increment equivalent to that obtained from coaching. 
The present study does not provide evidence on this point. 

Curr and Gourlay (1960) have proposed an alternate definition of 
coaching, not in terms of a procedure, as in the present study, 
but in terms of observed effects: 

(a) “Coaching is present when gains vary consi 
the type of test used to measure the basic skill to 
Such variation in gains we would regard as evidence more of a 


derably with 
be acquired. 


$ 
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variation in the acquisition of test-skills than of a genuine im- 
. provement in the basic skill.” 

(b) “A gain can be attributed to coaching when it is transitory, 
i.e., when it disappears in a fairly short period of time—12 months 
or less.” The gains in the present situation meet the second of 
these criteria, but as they were in general not confined to specific 
item types, they do not meet the first. In terms of Curr and Gour- 
lay’s definition, therefore; no coaching effects were found. 


Summary 


Coaching may increase students’ scores on an achievement test 
without increasing, or increasing only temporarily, their skills in 
the area being tested. This study deals with the immediate and 
longer term effects of coaching on the CEEB English Composition 
Test. 

Students at each of six cooperating high schools were randomly 
assigned to coached and non-coached conditions. Following the 
coaching sessions, all students took the English Composition Test. 

The immediate effects varied from essentially no mean differ- 
ence in test scores between coached and non-coached students at 
one school to a mean difference of 73 points favoring the coached 
students at another school. In general the coaching increment was 
not confined to the specific item types coached for. The coached 
groups achieved their superiority by getting fewer items wrong. 

When coached and non-coached students were retested ten 
months later it was found that the coaching increment had dis- 
appeared. Estimates of growth and practice effects from the test- 
retest data of the non-coached students varied from a mean of 
15 points at one school to a mean of 94 points at another. 

It was concluded that the candidate who receives coaching 
may indeed have an advantage over the candidate who does not. 
But the variability from school to school both of coaching effects 


and of score gains due to “growth” indicates that the specifics of 
the teaching situation are quite important. 
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SOME HYPOTHESES CONCERNING TEST ADEQUACY* 
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University of Chicago 


An earlier paper (Fiske, 1963) presented the rationale for using 
the cumulative homogeneity model in developing psychological 
tests, especially in the personality domain. It also outlined in non- 
technical form a proposed technique for assessing the adequacy of 
a test in terms of such a model. 

This paper describes the analytical technique more specifically. 
It also summarizes the findings from application of the technique 
to various scales from a wide variety of existing tests. From these 
findings are derived generalizations in the form of hypotheses. 
These state the characteristics of a test which are believed to con- 
tribute to high degrees of adequacy. 

The concern underlying this paper is measurement in the ser- 
vice of theory. The goal is to improve personality measurement 
so that personality concepts can be better studied and refined. The 
technical approach utilized for this purpose is unsophisticated. It 


involves simply a breakdown of the total observed variance in a 


set of test responses to determine the relative contribution of the 
for items, and also the remain- 


means for persons and of the means ; p 
ing variance from all other sources, including especially the inter- 


action between persons and items. A simple descriptive approach, 
has been employed to 


rather than one using variance estimates, 
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emphasize the particular observations actually in hand and the par- 
ticular measuring instrument. While the testing of a theoretical 
proposition requires inference from the sample measured to some 
loosely described population, we consider it dangerous to assume 
that any single personality test adequately represents the total 
domain of a personality construct. Stated in different words, 
most current personality constructs are broad and heterogene- 
ous; following the strategy based on the cumulative homogeneity 
model, any one test should be designed and used to assess only one 
specified portion of such a construct. Appropriate combinations of 
such restricted sets of measuring operations can then be formed 
to represent a total construct, or a major segment of it. In principle, 
this procedure should make possible a close fit between a group 
of measuring operations and a construct. Maximizing the con- 
gruence between construct and operations seems essential to the em- 
pirical study of each construct, for the purpose of furthering the 
science of personality. 


Assessing the Homogeneity of a Test 


Below is an informal statement of computational procedures and 
indices for assessing the homogeneity of persons, the homogeneity 
of items, and the proportion of unwanted variance in the responses 
of a sample to a set of test items. The underlying rationale is de- 
veloped in “Homogeneity and Variation in Measuring Personality” 
(Fiske, 1963). í 

The ideal is taken to be the following: 

1. Optimizing the variance of person means (this goal will usu- 
ally be equivalent to maximizing this variance since existing tests 
never approach the postulated ideal of a rectangular distribution). 

2. Optimizing the variance of item means (again, the postulated 
ideal of a rectangular distribution requires much larger variance 
than is ever found). 


3. Minimizing unwanted variance associated with instability and 
person-item interaction. 

The variance of person means is used rather than the more 
usual variance of total scores for persons because the former is 
independent of n, the numberof items. Similarly, the variance of 
item means is independent of N, the number of persons. For any 
test, the variance of person means can be compared to that for 
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item means. Also, the variances of person means (or item means) 
from several tests can be compared. Thus the adequacy of a test 
is evaluated in terms of the quality of the measurement ob- 
tained from the response of the typical person to the typical item. 
Once this quality has been judged as adequate, a decision about the 
suitable length of the test can be made in terms of such considera- 
tions as the desired level of the conventional index of internal con- 
sistency and of the amount of testing time available. 


The Method 


The analysis essentially consists of writing a data matrix, in 
which the rows are persons and the columns are items and each 
entry is the response of a person to an item, and of computing 
various sums of squares. These values are then used to obtain 
four coefficients and to break down the total variance into three 
parts. 

The total sum of squares and the sums of squares for rows, for 
columns, and for remainder are computed in the standard fashion 
for a two-way analysis of variance with one entry per cell. Divid- 
ing each of the latter three partial sums of squares by its degrees + 
of freedom yields the usual variance estimates, which provide a 
convenient way of obtaining certain coefficients. 

The standard coefficient of internal consistency, r+ (the general 
coefficient alpha [Cronbach, 1951]; for dichotomous scoring, Kuder- 
Richardson 20) is obtained by taking the complement of the vari- 
ance estimate for remainder divided by the variance estimate for 
rows. The dual of this coefficient, øg, iS obtained by taking the 
complement of the ratio, variance estimate for remainder over that 
for columns (cf. Guilford, 1954, pp- 383-385). 

More useful than these coefficients are the corresponding ones 
for items and persons. The coefficient rz can be stepped down by 
the Spearman-Brown formula to obtain the intraclass correlation 
for items (which can informally be seen as the average correlation 
between pairs of items). Similar treatment of fg yields Typ, the 
intraclass correlation between persons (or the average jntercorrela- 
tion between pairs of persons). 

In this approach, the variance indices are descriptive; they are 
stated in terms of actual variance in the sample (rather than as 


estimates of variance in the population). The total variance, 8:7, 


72 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


is the variance of the total distribution of entries of the data matrix. 
If the scoring is dichotomous (if only 0 and 1 are used as entries), 
the total variance is the proportion of 0’s times the proportion of 
T’s (i.e, pq). If the scoring is not dichotomous, the total variance is 
obtained by dividing the total sum of squares by nN, the total num- 
ber of observations. (This total variance should not be confused with 
the variance of person scores which is used in the standard formula 
for Kuder-Richardson 20.) 

By dividing the sum of squares for columns by nN, we obtain 
si”, the actual or obtained variance of the item means. Similarly, 
still dividing by nN, we obtain the actual variance of person means 
(8u*) from the sum of squares for rows, and the remainder vari- 
ance (s,*) from the remainder sum of squares. 

While the total variance for dichotomously scored tests usually 
is close to .25, it may be somewhat less if the group mean departs 
considerably from .5, the middle of the possible range of means. 
Moreover, it is desirable to be able to compare tests with dichoto- 
mous scoring and tests with multi-step scoring, where the total 
variance may be incommensurate with that from 0,1 scoring. 
Therefore the three component variances are stated as propor- 
tions of the total variance. 

It should be noted that for a test with dichotomous scoring, it 
is not necessary to write out, or even to have available, the data 
matrix. The necessary computations can be performed from a 
distribution of person scores and a distribution of item endorsement 
values. The total variance is obtained from the grand mean as indi- 
cated above. 

The computations described above provide an extensive set of 
indices for assessing a test in terms of the cumulative homogeneity 
model. The description of these operations is also intended to 
show certain mathematical relationships among the indices, If in 
actual practice, one is interested only in certain indices, some of 
these computations may not be necessary. 


Indices of Test Adequacy 


The most important single index is the proportion of remainder 
variance, 8,”/s;*. This value indicates the total relative contri- 
bution of what is often called “error.” We use the term “remainder” 
rather than “error” because we can specify some of the elements 


it 
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included in this value. By subtracting the contributions of per- 
sons and of items from the total variance, we have a value which 
is the sum of the contributions from several unwanted sources. 
While these cannot be separately assessed with the data from a single 
trial, something is known of their approximate size from data 
with repeated trials. 

In dichotomously scored tests, one part of the remainder variance 
is an artifact of such scoring. For any corresponding arrays of 
item and of person means, the expected value for each cell can 
be computed. The artifactual component is the mean squared devia- 
tion of such values from 0 or 1, whichever is closer. Thus the arti- 
fact stems from the restriction of the scoring to these two values 
(to a discrete scale) when a completely homogeneous data matrix 
requires that the cell entries be on a continuous scale. In the ideal- 
ized case with rectangular distributions of person and of item 
means, this artifactual component reaches .33 as its relative con- 
tribution. It approaches .00 as the variance of either rows or of 
columns approaches zero. The more steps that are used on the scor- 
ing of each item, the less the contribution of this artifact. 

Another large component in the remainder variance is person- 
item interaction. This source reflects the extent to which persons 
give responses to items which are different from those expected on 
the basis of the corresponding row and column means. In this anal- 
ysis, we have already identified part of this average discrepancy as 
artifactual. The rest of this discrepancy can be attributed to true 
person-item interaction. 

Conceptually, such interaction has two parts. One part may be 
attributed to variability of responses over time, or instability. Anal- 
yses of some data from repeated trials suggest that the pro- 
portional contribution from instability is relatively small. I.e., the 
departure of the response of a person to an item from his mean 


response over a series of trials contributes relatively little to the 
be either toward the value 


total variance. Such departures may 

predicted from row and column means or away from it. 
The second and, conceptually, the most important part of 

person-item interaction is idiosynerasy, the tendency of the person’s 

response (or average response over trials) to be different from that 

predicted, This source is heterogeneity, the complement of what 

we seek in maximizing the homogeneity of persons and of items. 
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Such idiosyncrasy may be explained in various ways. It may stem 
_ from a highly unusual, individual interpretation of an item. It may 
also come from the variable, as defined by the test items, taking 
different forms in different persons. E.g., while, for the group as 
a whole, Item A may be endorsed more frequently than Item B, 
there may be a minority of individuals whose response tendency of 
the Item A variety is much weaker than their tendency along the 
lines of Item B. 

The other two important pieces of information obtained from 
this analysis are reflected in the proportions of total variance con- 
tributed by the variances of item means and of person means. 
With dichotomous scoring, each of these values is .33 in what is 
postulated to be the ideal case. The relative variance of persons 
tells how well people are differentiated. The relative variance of 
items indicates the extent to which the items tap different amounts 
of the underlying variable, rather than clustering together near the 
average amount of the variable. Appreciable variation in item 
means is necessary to determine the comparability of response 
patterns for different persons. 

In practice, it seems desirable for these two values to be of about 
the same magnitude. The approach taken in this paper requires 
that neither one be increased at the expense of the other. It is 
obvious that a test with considerable item variance is of little value 
if it does not differentiate among persons. On the other side, it is 
asserted that differentiation among persons should not be em- 
phasized at the cost of item variance. 

If the relative variance of item means is low, it can be increased 
by adding items with endorsement values as different as possible 
from those of existing items, to make the overall distribution more 
closely approximate a rectangular one. If the relative variance of 
person means is low, the test is not differentiating well for the 
given group of persons to whom it was administered. It may be 
that the test is inappropriate for the group’s range of response 
strengths for the variable being measured. Or it may be simply a 
poor test. 

The same essential information is, of course, contained in the 
actual variances of item and of person means. 

If one prefers to study correlation coefficients rather than vari- 
ances, 74 and Tp) can be examined. With dichotomous scoring, each 
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of these two coefficients is .50 in the ideal case. tu is closely related 
to the variance of person means and Tp, bears a parallel relationship 
to the variance of item means. When n=N (when the number of 
items equals the number of persons), the relation between the mag- 
nitudes of the variances for person and item means (smp? and Sa”, 
respectively) is reflected in the relation between ry and Tpp. 

As an index of the usual notion of internal consistency, the con- 
sistency of items, fu is preferred to rs because the latter varies with 
the number of items. re is an evaluation of the given set of items 
as put together in the particular test. Tu indicates the adequacy of 
items of this kind for the group of persons tested. 

Tpp indicates the extent to which the several people are produc- 
ing the same pattern of responses, the pattern summarized in the 
ordering of the item means. In the ideal case, each individual’s 
pattern will be consistent with that of every other individual and 
with that of the item means. Each person’s pattern will have a 
general elevation determined by his mean score. This score and the 
scoring system may produce ties between items with different 
group means but will not produce reversals. Thus, for dichoto- 
mous scoring, each person’s pattern will match that for each other 
person, except for a run of items, the length of which is equal to 
the difference between their total scores. 

Tgog is of negligible value since, for any test, it varies with the 
number of subjects contributing to the data matrix. 

(For a somewhat similar approach to the evaluation of a test, 
see Cronbach, Rajaratnam, and Gleser, 1963. The major differences 
are these: Cronbach and his associates are working from a relative 
frequency model rather than a cumulative homogeneity model; 
they use variance estimates rather than straight descriptive or 
obtained variances.) 


Result and Interpretations 


The Data 


Table 1 summarizes most of the applications which have been 


made of these procedures for analyzing homogeneity. Where two 


or more scales from one instrument have been studied, the scales 


with the highest and the lowest proportional remainder variance 


(s/s?) are usually presented. The various instruments are listed 


TABLE 1 


Homogeneity Indices for Various Scales 
Smi? Sup sè 
Test and Scale Neus tilts. T T eA 
Edwards Personal n Deference 30 .36 03 07 .09 .06 .85 
Preference Schedule n Achievement 80 .63 .02 08 .10 .05 84 
Social Desirability 86 .93 05 08 10 .07 83 
(Edwards) 
Kuder Preference Literature 50 47 05 14 .l4 .06 .80 
Record Mechanical 50 31 COT 17 E 07 20 
California Psycho- Achievement via 
logical Inventory Independence 34 .76 05 09 10 .06 .84 
Good Impression 34 .38 .09 .27 .27 .08 .65 
Allport-Vernon- Theoretical (1-4) 20 2.86 .07 .12 15.10 .75 
indzey Political (1-4) 27 2.29 04 19 .20 .07 .72 
Thurstone Temper- Emotionally Stable 30 .46 .03 11 13 07 20 
ament Schedule Sociable 40) 9.67) 512 1.274 26. 11 68 


Personality Re- Attitude toward 


search Inventory work 82.53 10 05 07 .18 .75 
(ETS) Talkative 32 .40 .24 12 10 .28 61 
WAIS Comprehension 
(0-2) 48 168 .06 11 11 11 .78 
Similaritjes (0-2) 48 1.51 .13 34 .30 .13 .56 
Manifest Anxiety 
Scale 30 .40 .21 16 .15 19 .66 
MMPI Hy (Pre-therapy) 30 .42 01 .20 .23 .02 .75 
Hy (Peace Corps) 36 .37 .01 .56 .57 .01 42 
Holtzman Ink- Hostility (0-2) 20 .29 03 07 10 .12 .78 
Blot Human (0-2) 20 84 03 85 385 .08 .57 
Conation Indicator Science 60.57 17 20 117 4.24 «60 
Daring 50.52 417 84 .28 20 .52 
Street-Gestalt 50 .60 10 31 .30 .08 .62 
Physiognomic Cues (0-5) 50 1.80 22 38 .33 .16 51 
Weitzenhoffer-Hilgard Hypnotic 
Susceptibility 124 44 29 23 17 29 54 
ACE Verbal Completion 50 .69 .07 (32 32 07.61 
} Analogies 50 .66 07 57 4.55 .04 «AL 
Self-Disclosure to Close Male 
Questionnaire Friend (0-2) 16 144 .26 23 23 23 54 
to Casual Ac- 
quaintance (0-2) 39 .86 40 44 32 28 40 
FIRO Expressed Inclusion 76 .51 25 40 32 23 .45 
Wanted Control 76 .50 28 54 43 20 37 
S-R Inventory Total Test (1-5) 71 2.05 12 40 .38 08 .54 


of Anxiousness 

Mode 8—(Not) Enjoy the Chal- 
lenge (1-5) 71 268 16 12 =. ‘ 

Mode 1—Heart beats faster (1-5) 71 2.70 31 48 33 25 42 
Situation 7—Sailboat on rough sea 
i q (1-5) 71 1.91 28 24 19 27 55 
Situation 4—Ledge on mountain (1-5) 71 2.58 87 42 30 29 41 
Situation 4—Four items for Distress 


factor (1-5) 71 3.18 59 37 16 58 .26 


* Means greater than unity can occur for tests with multi-step (rather than dich tomous) scor- 
ing of items. Where such scoring was used, the score range is given after the name of the senle, 
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in descending order of this expression for the remainder variance 
(using the average value when two scales from the same instru- 
ment are presented). Omitted are various data for repeated trials 
and data from a few instruments for which only one scale was 
analyzed. 

The remainder variance is stressed in this table and in the ensuing 
discussion because it is believed to be the best single index of the 
adequacy of a test. While differentiation among persons is highly 
desirable, minimal person-item interaction is essential for a good 
instrument designed to measure individual differences. 

Data for two groups were available in some instances. When 
the two groups were highly comparable, the indices were typically 
very similar. When the conditions of testing or other factors made 
the groups not comparable, the corresponding indices differed to a 
greater or lesser extent. The largest observed differences, for the 
MMPI Hy scale, are given in the table. 

The ACE, the Street-Gestalt, and the Physiognomic Cues were 
given with time limits. However, the great majority of the subjects 
attempted almost all of the items in these scales. 

The data selected for Table 1 may yield an erroneous impres- 


sion that there is a strong association between fe and Tp», and 


also between sy42 and smp» For unselected tests, such associations 
over tests are much smaller. 

Inspection of Table 1 discloses a great range in each of the 
several indices: not only does the proportion of remainder vari- 
ance vary from .85 to .26, but the proportions of variance associated 
with item and with person means vary from 09 to .57 and from 
04 to .58 respectively. The smallest and largest values of the co- 
efficients are .01 and .59 for items and .05 and .57 for persons. The 
problem is to infer the sources of these large variabilities. 


The Degree of Structure in the Task Given to the Subject 


n appears to be the degree of structure 
r the subject. There seems to be some 
of structuring and the 


One pertinent conditio: 
in the task which is set fo 
negative relationship between such degree 
contribution of remainder variance. 

In intelligence tests and in many cognitive-perceptual tests, the 
subject has a clear instruction about what he is to do and about the 
criteria he is to use in selecting his response. The standard per- 
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sonality questionnaires, on the other hand, provide a task with 
very little structure (cf. Fiske and Butler, 1963). The subject has 
two difficulties. First, he has vague criteria for determining what 
his response should be: for example, how often is “often”? Second, 
the process by which he arrives at his response, before coding it 
into the available alternatives, is not indicated explicitly. For in- 
stance, the question might be: do you find it difficult to give a 
speech before a large group? The subject can go about the process 
of answering it in several ways. He can compare this statement to 
his general picture of himself, He can recall one or several perti- 
nent experiences and base his response on his memory of his feel- 
ings. He can also decide that an affirmative answer would be true 
for most people and therefore is true for him. (These are some 
of the processes discerned in reports from subjects responding to 
such an item.) In8ofar as various subjects go about answering such 
a question in different ways, the task is not structured. 

Another way of interpreting this condition is to consider whether 
the subject’s task is almost wholly to react to present stimuli or 
Tequires the subject to scan his memories for pertinent instances of 
past behavior or feelings. Intelligence and cognitive-preceptual 
tests involve immediate processes and reactions while most ques- 
tionnaires require mediation of the response process through prior 
experiences and memories, A seeming exception to the generaliza- 
tion that such a distinction is related to adequacy of the instrument 
is the $-R Inventory (Endler, Hunt, and Rosenstein, 1962). While 
its quality involves other considerations, the merit of this instru- 
ment may also stem in part from the fact that, for many of the 
presented situations, the subject must imagine himself in them 
since they are ones which few people have actually experienced. 

When a task is structured for each subject, it is more likely to be 
approached in the same way by the several subjects. The greater 
the homogeneity or similarity between the subjects’ approaches to 
a test, the more adequate the test in terms of the criteria used in 
this paper. This proposition is, at present, an hypothesis which has 
not been systematically tested. 

For a given subject, the structuring of a task can be increased by 
familiarity with it. A person who has experienced a given test has 
had the opportunity to develop attitudes toward it. Such acquisi- 
tion of stable perceptions and reactions to a test probably accounts 
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for the typically slight gains in adequacy of the various indices on 
second and later trials. (Only limited investigation of this effect 
has been carried out.) 

A special case of structuring is found in the Hypnotic Suscepti- 
bility Scale (Weitzenhofier and Hilgard, 1959). The several items 
in this scale are used to estimate the degree of trance present in 
the subject over the several minutes required for application of the 
test stimuli. This unique instrument highlights the way in which 
structuring contributes to homogeneity: in a structured task, the 
subject tends to approach the several items in the same way. The 
critical factor is this maintenance of set across items. 

From these considerations, we derive the following hypothesis: 
1. The adequacy of a test is a direct function of the degree of 
structuring of the task, structuring which is established for the sub- 
ject by the instructions and the situation. 8 vj 

We have not defined “structure” yet. It is a concept used here to 
refer to a common element in the several aspects of the testing 
situation considered above. Structuring imposes constraints on the 
subject so that, in the ideal case, the responses of each subject are 
determined solely by the degree to which he possesses the disposi- 
tion being measured. To delineate the term more sharply and with- 
der it in two ways. A task is struc- 


out circularity, we must consi 
tured when, from the subject’s point of view, he feels he knows 
í the testing experience are 


(a) what the potential consequences © nce £ 
for him, what effect (if any) his performance in the situation 
expected to do. This 


will have upon him and (b) what he is 


latter expectation includes not only the way he is to approach each 
between the item and him but 


item-stimulus and the transaction 
also the criteria he is to use in selecting his response. From the 
examiner’s viewpoint, a task is structured when the several sub- 


jects agree on the nature of the potential consequences of the 


testing and on these expectations about the task for the subjects, 
same demand characteristics 


i.e., when the subjects perceive the 
(Orne, 1962) in the situation. Hypothesis 1 refers to both sub- 
jective and objective forms of structuring. ANH 

As so defined, the degree of structure in a testing situation can- 
not be directly assessed by any one set of operations. Significant 
indirect evidence can be obtained from the consistency of the 
responses over items and persons and valuable information can 
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also be sought through carefully planned questioning after the test 
is completed. 

The classification of immediacy versus mediacy, mentioned earl- 
ier, overlaps greatly with a classification proposed by Campbell 
(1957): the objective versus voluntary nature of the criteria to be 
used by the subject in selecting his response. Both classifications 
can be subsumed under the concept of structure. While tests with 
objective criteria are more structured, the large range of structur- 
ing among tests with voluntary criteria indicates the considerable 
contribution of other conditions such as immediacy-mediacy and 
those discussed below. 


The Degree of Structure in the Item-Stimulus 


Another condition contributing to test adequacy appears to be 
the degree of structuring present in the items. Such structuring 
can be present in two places: the question or stimulus to which 
the subject reacts and the response alternatives afforded him. The 
latter will be considered in the next section. 

The stimulus in an intelligence test can be said to be highly 
structured, In this instance, structure is defined as univocality 
or lack of ambiguity, and can be assessed empirically. Items in in- 
tellegence and perceptual-cognitive tests have the same meaning to 
those who can give the correct answer, In contrast, the typical 
item in personality questionnaires is subject to diverse interpreta- 
tions (Benton, 1935; Eisenberg, 1941; Loehlin, 1961). However, 
items in personality instruments can be highly structured by in- 
creasing the degree of specification. Thus the Jourard Self-Dis- 
closure Questionnaire (Jourard and Lasakow, 1958) specifies not 
only the content of the possible disclosure but also the person to 
whom it is disclosed. The S-R Inventory asks for a report on the 
extent of each response tendency in each specified situation. To 
be sure, a composite score from such tests is affected by consider- 
able interaction between persons and objects or between persons 
and situations. However, such an instrument has the potential ad- 
vantages of providing specific subscores for objects or situations 
and of enabling one to estimate the extent of such interaction. 

Lumping together explicit specification and lack of ambiguity 
in meaning, we formulate this hypothesis: 2. The adequacy of a test 
ts a direct function of the degree of structuring of the typical 


pan 
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item-stimulus. Structuring from the examiner’s point of view can 
be determined in terms of the degree of consensus among the 
subjects in their perceptions or interpretations of the items, as re- 
ported under a separate set of instructions. To assess the extent to 
which the subject sees the item as structured, it is probably suffi- 
cient simply to ask him whether he felt any uncertainty about what 
the item meant. The first, objective type of structuring is neces- 
sary for test adequacy. The second, subjective form is probably 
desirable although it is not sufficient: even though subjects may 
not experience any uncertainty about an item, they may interpret 
it in different ways. 

The data for the S-R Inventory reported in Table 1 illustrate a 
trend which may turn out to have very great significance. When 
the responses for each specified situation are analyzed as separate 
“tests,” ry and Smp? (and also Sup?/s:2) typically are considerably 
higher than the corresponding values for the total test. Preliminary 
data on an experimental test which also specifies situations show a 
similar picture. When each type of response is analyzed separately, 
the same indices tend to be more favorable but the other set of 
indices (rp and sm?) do not systematically improve. (In the §-R 
analyses for separate Modes of Response, the median values of 
these latter indices are substantially lower than those for the test 
as a whole.) 

The interpretation of these comparisons remains to be estab- 
lished. It may be that, in such questionnaires, the delineation of a 
specific situation enables the subject to keep a single clear mental 
image in mind as he selects his responses to the several items. It 


may also be that behavioral dispositions are associated with (or 


determined by) the particular situation to a much greater degree 
tual formulations. Whatever the 


than is recognized by most concep’ 

explanation, the data strongly suggest that the variance of persons 
can be increased, and at least questionnaires can be made more 
adequate, by incorporating in groups of items a single explicit set 
of situational conditions. 


The Degree of Structure in the Response Alternatives 


One might consider the possibility that the adequacy of a test is 
in part a function of the degree of structuring of the responses 


which the subject is permitted to make, Here again, structure 
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means univocality. However, there is little evidence to justify 
viewing this aspect of a test as a separate property related to 
adequacy. It seems pertinent, for example, that the S-R Inventory 
must be viewed very favorably in terms of the criteria being used 
in this paper in spite of the fact that the subject is given a set of 
five ordered steps from which to select his responses, with only the 
first and last step defined at all, and these definitions being very 
general for the high ends. 

More structured response alternatives are ordinarily found when 
the task is structured, as in intelligence tests, and when the items 
are structured, as in perceptual-cognitive tests. They also are more 
likely to occur when the criteria for selection of responses are 
objective rather than voluntary. 


The Degree of Substantive Homogeneity 


Underlying the methodological approach presented in this 
paper is the proposition that the adequacy of a measuring procedure 
used in research on a construct should be defined in terms of the 
substantive homogeneity of the test over items and over people. 
The substantive homogeneity of a test refers to the degree to 
which the test scores reflect a unitary conceptual entity. It is maxi- 
mal when the same function or Tesponse tendency is measured 
by the several items in each of the subjects. The several criteria are 
proposed as indices of such homogeneity. The two hypotheses above 
state potential conditions contributing to this homogeneity: the 
structuring of the task elicits the same function in the several 
subjects as they cope with each of the items; the structuring or 
Specificity of the items makes it possible to utilize items pertain- 
ing to a common response tendency. 

Separate attention is being given to this basic concept because 
the item content is of overwhelming importance. The data in 
Table 1 suggest conditions contributing to substantive homoge- 
neity. Among the more adequate tests are those in which a priori 
identification of content was given major emphasis. (This associa- 
tion was first observed by Lee J. Cronbach.) The S-R Inventory 
was formulated with no use of the techniques of test development 
and refinement. Psychometric properties played a secondary role 
in the construction of FIRO-B (Schutz, 1958). In contrast, em- 
pirical criteria for item selection were the sole or primary consider- 
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ations in the development of such tests as the Kuder Preference 
Record, the California Psychological Inventory, and the MMPI. 

Factor analysis undoubtedly can be used to improve homo- 
geneity (Comrey, 1961). The most favorable total pattern of in- 
dices found to date is that for one situation in the S-R Inventory 
and for four modes of response with high loadings on the “distress” 
factor emerging in the work of Levin (1963). On the other hand, 
factor analysis was used in the development of the Thurstone 
Temperament Schedule, an instrument which is only moderately 
successful in terms of our criteria. 

Part of Table 1 suggests that competition between different vari- 
ables in the same item is undesirable. In the EPPS, the two state- 
ments in each pair represent different needs. Each item in the first 
part of the Allport-Vernon-Lindzey requires the choice between 
two values; in the second part, four values must be ordered. 

Notwithstanding the definitional aspect discussed above, the 
primacy of substantive homogeneity justifies the formulation of 
this proposition as & hypothesis: 3. The adequacy of a test is a 
direct function of the substantive homogeneity of tts items. 

Some readers will argue that the maximizing of substantive 
homogeneity leads to the creation of scales which assess & very 
limited conceptual domain. Just as attitude scales with high homo- 
geneity and reproducibility by Guttman’s criteria tend to be nar- 
row and redundant, so highly homogeneous tests may appear so 
specific that what they measure may seem trivial. Thus in several 
of the FIRO scales, the same stem is typically used for two items 


which differ only by presenting alternative sets of response choices 


(one set in terms of temporal frequency and the other in terms ‘of 
subject manifests the tend- 


the number of people with whom the i 
ency stated in the stem). Similarly, the S-R Inventory has its 
best sets of indices when some one situation or single mode of re- 


sponse is analyzed separately. 

Tt seems probable that high homogeneity, 28 assessed by the pro- 
posed criteria, will typically be associated with a limited diversity 
of item content, Such an association, however, need not be an 


obstacle. The best strategy for optimal measurement of a per- 


sonality variable involves several steps. First, the variable must be 


analyzed conceptually to determine the various forms and the var- 


jous situations or contexts in which the behavioral tendencies ap, 
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pear. Then scales must be constructed for each form, explicitly 
covering the relevant contexts. Once the homogeneity of such 
scales has been established, the experimenter is free to utilize one 
or more of these scales in basic research on that variable. When 
several scales are used, separate analysis for each one would seem 
desirable, in addition to analyses of any composite scores obtained 
by conceptually appropriate combinations of certain scale scores. 
Empirical tests of this strategy are being initiated. 

The objection may also be made that a structured test with 
homogeneous content will often enable the subject to see readily 
what variable is being measured. This possibility is especially 
marked for personality questionnaires. Any undesirable effects 
from such obviousness can be somewhat reduced by the usual prac- 
tice of administering the items from several scales in mixed order 
within a single instrument. 

The more obvious the content, the greater the potential contri- 
bution of such dispositions as the tendency to be defensive or to 
present oneself in a socially desirable light. The problem of assess- 
ing and controlling such tendencies is beyond the scope of this 
paper. To the extent that they are uncorrelated with the variable 
for which the test is designed, they will reduce homogeneity and 
the indices discussed above. If these dispositions are substanti- 
ally correlated with the target variable or if they contribute most of 
the systematic variance, the test may appear fairly adequate in 
terms of the proposed criteria. Such undesirable conditions should, 
however, be detectable in appropriate validational studies. 


Further Considerations 


The Group Mean. Close examination of the data has failed to 
reveal any systematic contribution of the grand mean for all cells 
in the data matrix. To be sure, for most of the scales that have 
been analyzed, the mean falls near the middle of the possible 
range of scores. There is a slight suggestion that scales with 
more extreme means tend to have relatively higher proportions of 
remainder variance. 

Dichotomous versus Scaled Scoring of Items. As noted above, the 
use of dichotomous scoring introduces artifactual variance. Anal- 
ysis of an ideal data matrix indicates that this contribution can be as 
much as one-third of the total variance. Analyses of other con- 
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trived homogeneous matrices not having full rectangular distribu- 
tions of item and of person means yield contributions from that 
value down to 25 per cent. 

Now completely satisfactory procedure is known for estimating 
this component in empirical data. One possibility is to take the 
item means as fixed and from them to determine the maximum 
possible variance of person scores (ef. Carroll, 1945). The sum of 
this latter quantity and the variance of the item means is then sub- 
tracted from the total variance. By this method, the estimated 
contribution from dichotomous scoring is typically around .06 and 
07, or roughly 25 to 30 per cent of the total variance (for various 
dichotomously scored tests). 

Another approach is to compute the minimal amount of this com- 
ponent for the obtained set of item and person means, i.e., the 
amount which would have been obtained if each person had 
given the response predicted by the corresponding person and item 
means. For a few tests with dichotomous scoring, values in the 
order of .06 to .11 have been obtained. These are 46 to 75 per cent 
of the residual variance (the higher per cent when the residual 
is small in absolute size) and 27 to 43 per cent of the total variance. 

Neither rationale appears to be satisfactory. The former yields 
the value which would have been found if, contrary to fact, the 
subjects’ responses had been such as to maximize the variance of 
their scores. The latter analyzes the hypothetical case in which 
the person and item means remain as observed even though each 
person’s response to each item is completely consistent with the over- 
all pattern of item means: it is obvious that if such consistency 
were present, if the Jatter condition were met or closely approxi- 
mated, the variances of item and person means would be greater. 

Another way to approach the matter is to ask whether tests with 
multi-step scoring have appreciably better indices than they would 
have had if they had been scored dichotomously. We have applied 
the standard analyses to 19 scales which had multi-step or scaled 
scoring and compared the indices from this original scoring with 
those from dichotomous scoring, dividing the scoring scale near the 
group mean. The median increase in the relative remainder vari- 
ance (s,2/s2) is 06. The median decrease in the proportions of 
variance associated wi 


th item and person means is 03 for each. 
While these effects are only in the order of one-tenth of the 
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original values, they vary considerably from one test to another, 
and some of the largest effects occur for scales with quite favor- 
able indices when scored with multiple steps. Thus it appears to be 
desirable to use more than two scoring steps whenever the con- 
tent is appropriate. 

The above discussion should not be taken as suggesting that 
the use of three to five steps in item scoring eliminates the scoring 
artifact completely. To obtain a completely homogeneous data 
matrix with considerable item and person variance and with no 
artifactual variance, it would be necessary to have more scoring 
steps than the number of steps between the highest and lowest 
persons when their scores are stated in conventional terms as the 
sum of the item scores (rather than as the mean item score). The 
use of several scoring steps somewhat reduces but does not eliminate 
the artifactual component. 

The Use of Stratified Items. Some attempt has been made to ex- 
plore the possibility of improving tests by selecting from existing 
tests a set of stratified items, a set yielding a rectangular distri- 
bution extending over a major part of the potential range from 
.00 to 1.00. The results have not been encouraging. While some of 
the indices become more favorable in some instances, such a tech- 
nique for test refinement is no panacea. The findings suggest that 
this procedure does little or nothing to reduce the major source 
of difficulty, person-item interaction. They also suggest the tenta- 
tive inference that significant improvement in the several indices 
would not be obtained by adding further items of the same kind 
to a test for the purpose of extending the range of endorsement 
values and increasing the variance of item means. 

Idiosyncrasy Scores. For each cell in the data matrix, we can 
determine the response expected on the basis of the row (person) 
mean and the column (item) mean and compare it with the ob- 
served response. Thus if the test is scored dichotomously and the 
expected value is above .5, the response should be 1; if below, it 
should be 0. If the observed response is not as predicted, we can 
classify it as idiosyncratic. We can then count the number of such 
responses for each person and call this his idiosyncrasy score. Simi- 
lar scores can be obtained for the several items. 

It was thought that such scores might be useful for refining 
tests. For example, items with high idiosyncrasy scores should be 
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eliminated. However, the same type of improvement can be ob- 
tained by standard methods, such as item-test correlation. In princi- 
ple, such scores might be used for identifying persons who seemed 
to be approaching or interpreting many of the items in a highly 
individualistic way. Whether such identification could have any 
practical utility remains to be demonstrated. 

There seems to be little of value in such scores. They have 
very little internal consistency when they are analyzed in the same 
fashion as the conventional response data. The sole consistent 
finding is a negative relationship between idiosynerasy score and 
distance of the person or item mean from .5 (for tests with dichoto- 
mous scoring): especially for items, the more extreme the mean, 
the lower the idiosynerasy score. This finding would appear to be 
one further argument for constructing tests with rectangular dis- 
tributions of item means rather than with means clustering near .5. 


Summary 


Operations are persented for evaluating the adequacy of a test in 
terms of a cumulative homogeneity model. Particular emphasis is 
placed on the proportion of the total variance in a matrix remaining 
after subtraction of the variances of item means and of person 
means. This remainder variance includes the contribution of arti- 
factual variance from dichotomous scoring and interactions between 
person and item. 

Analyses of existing personality tests reveal much variation in 
adequacy. As guides for test construction, three hypotheses 
are proposed: The adequacy of a test is a direct function of (a) 
the degree of structuring of the task established for the subject by 
the instructions and the situation, (b) the degree of structuring of 
the typical jtem-stimulus, and (c) the substantive homogeneity of 


its items. 
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PATTERNS OF MOTIVATION FOR 
COLLEGE ATTENDANCE: 


ARLENE G. COHEN ann GEORGE M. GUTHRIE 
The Pennsylvania State University 


Tu experience of counselors and the evidence of research do 

not agree on the role of motivational factors in academic perform- 
ance, Contact with students leaves one with the firm conviction that 
motives to achieve have much to do with how well students per- 
form. Given the same level of ability one student may do well, 
another poorly, and the difference seems clearly related to emo- 
tional or motivational differences in the students. Because this ob- 
servation is made every day there have been many attempts to 
include measures of motivation, personality, and adjustment in pre- 
diction batteries. However, the improvement in prediction with 
the addition of non-intellective predictors has rarely exceeded .05, 
although there was room for improvement since intellective pre- 
dictors produce an average multiple correlation of +55. (Fishman, 
1962) 
It is our impression that the addition of non-intellective variables 
to prediction formulas has not improved prediction appreciably 
because the relationships between variables assumed by multiple 
regression equations are not the same as those assumed by clini- 
cians. Multiple regression equations presuppose linear compensa- 
tion whereas the clinician operates with a sort of typology in which 
he sees individuals in certain categories rather than at certain points 
along continuum. As Meehl’s summary (1954) suggests, however, 
clinicians have not demonstrated any remarkable ability to 
capitalize on their more flexible model. 

1 This report is based on & thesis supervised by the second author and sub- 
mitted by the first author to the Pennsylvania State University, 1964. 
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In spite of the failure of multiple regression procedures to make 
productive use of motivational factors there are alternatives to the 
crystal ball. Stern, Stein and Bloom (1956) have outlined and 
evaluated a number of methods of coping with the multidimension- 
ality of the criterion and have suggested improved methods of 
combining predictor variables. Saunders (1956) has developed a 
mathematical expression for moderator variables, variables which, 
although uncorrelated with either predictor or criterion, enhance 
the level of prediction by separating less readily from more readily 
predicted subjects. Middleton and Guthrie (1959) have domonstra- 
ted that there are different personality syndromes among students 
who have achieved at the same academic level. Taking these 
studies together suggests that a desirable tactic would involve cate- 
gorizing subjects and developing separate prediction equations 
for each homogeneous group. This would result in the use of 
different beta weights and it would lead to the use of different 
predictor variables as well. 

In this research we are concerned with motivations for attending 
college which we feel are fundamental among non-intellective de- 
terminers of achievement. There have been many studies of mo- 
tives for college attendance. In one of the more extensive surveys, 
Iffert (1958) found an emphasis on vocational reasons, with aca- 
demic, social service, personal, and family tradition considerations 
following in approximately that order. In their review of motiva- 
tional factors, Douvan and Kaye (1962) have emphasized voca- 

` tional preparation especially for males, and social mobility. They 
point out that motivations differ with different social levels. Havig- 
hurst (1960) has also emphasized the role of social status striving 
as a factor in college attendance. However, there has been little 
systematic study whether different patterns of motivation lead to 
different levels of achievement given the same level of ability. The 
motives for attendance and the drives for achievement have been 
examined in detail. But there has been little ingenuity in combining 
these with intellective factors to obtain improved prediction. 

Since high school grades reflect the unique combination for each 
individual of intellective and non-intellective factors, it is not sur- 
prising that they remain the best predictors of college perform- 
ance. Bloom and Peters (1961) controlling the error variance due to 
differences between high schools, and between colleges as well, 
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have obtained cross-validated correlations above .70 between high 
school and college performance. Reducing the variability in social 
influences by sampling permits an improvement of prediction. It 
may also be possible to improve prediction by accounting for social 
and motivational factors by measurement and classification. 


Method 


The purpose of this research was to identify patterns of motiva- 
tion among male undergraduate students. It was hypothesized that 
there is more than one pattern of motives and that groups of 
students could be identified with more or less homogeneous stated 
sets of purposes. We sought to identify these groups by inverse 
factor analysis, a method in which persons are correlated with one 
another and the factors derived are hypothetical modal persons, 
each one representing a sort of type characterized by a more or 
less unique configuration of motives. 

The correlations were based on the Ss’ response to a 105 item Q- 
sort with a forced normal nine-point distribution. The 105 items 
were selected from Iffert (1957) and from the statements of stu- 
dents made in response to a question about their reasons for going 
to college. A preliminary list of 140 items was administered in a 6- 
point form to 64 male undergraduates. The list was reduced to 105, 
shown in Table A,? by deleting items which were frequently 
omitted or which showed little variability between Ss. Each item 
contained a positive statement of one purpose for attending col- 
lege. r 

The final form of 105 items was typed on individual cards and 
each S was asked to sort so as to conform to a 9-point normal dis- 
tribution. The correlation between each pair of Ss was computed 
over the 105 items. i 

Our Ss were 200 male undergraduate students enrolled in Intro- 
ductory Psychology representing many fields of major interest ' 
with a concentration in business and engineering. One-half of the 
Ss were beyond their freshman year. The Ss were divided into two 
groups of 105 and 95, with the second group used to validate the 
findings for the first group. 

A 105 X 105 matrix of correlations was generated. Ten factors, 
te of the number of interpretable 
acted and rotated orthogonally 


w 
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using the Varimax solution. Correlations were computed between 
each item and each factor and the factors were interpreted by se- 
lecting those items with the highest correlation with each factor. 
Parallel data from the second 95 95 matrix were used to evaluate 
the stability of the findings on the first group of Ss. 

The naming and interpretation of factors is a matter of judgment, 
particularly where inverse factor analytic procedures are used. 
This is especially the case when the variables are people who 
steadfastly refuse to appear in factorially simple form. The degree 
to which the replication produces similar factors is also a matter of 
judgment. 


Results 


The first factor accounted for 29 per cent of the variance, the 
second for 9 per cent, and all ten accounted for 59 per cent of the 
total variance. On the replication the pattern was essentially the 
same with the factors accounting for 58 per cent of the total vari- 
ance. The two 10 X 105 matrices of correlations between items 
and factors are generated. Items correlating above +.30 with 
each factor are shown in Tables B-U2 : 

The results indicated that many Ss had significant loadings o 
more than one factor. The number of Ss loading .30 or more on a 
factor varied from 5 to 45. Communalities varied from .40 to .80 
with the majority falling between .55 and .75. The rotated factors 
were essentially unipolar with the exception of an occasional S who 

` showed a negative loading on a factor when all other Ss who had 
significant loadings were positive. When items are correlated with 
factor loadings we find both negative and positive correlations 
Suggesting that a factor is characterized by the acceptance of cer- 
tain values associated with the rejection of other values. This will 
become clear in the following example. 

In order to clarify our interpretation procedures we have shown 
in E 1 and 2 the items which correlate highly with Factors I 
and II. 


Reading the items which correlate with Factor I gives one the pic- 


2 Tables A-U have been deposited with American Documentation insti- 
tute. Order Document No. 8571, eane Poo for 35 mm ioia or S5 
for 6 by 8 in. photocopies. Order from ADI Auxiliary Publication Project, 
Photoduplication Service, Library of Congress, Wash. 25, D. C. Make checks 
payable to Chief, Photoduplication Service, Library of Congress. 
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TABLE 1 


Items Correlating with Factor I Desire for Intellectual Development 
with Denial oj Interest in Financial and Social Gain ` 


Item Correlation Content 
` Positive Correlations 
34 .59 I want to broaden my overall viewpoint. 
21 .58 I want to learn to think better. 
5 .53 I want to find out about certain areas of knowledge. 
101 49 I want to be of service to society. 
37 .48 I want to get the answers to many questions. 
102 48 I want to stimulate my interest in learning, 
6 AT I want to learn to think creatively. 
45 44 I want to know more about the world I live in. 
17 42 I want to go so that I can fulfill my purpose in life. 
8 42 I want to develop my abilities further. 
32 40 I want an education so that I can be more of a credit to 
my community, state and nation. 
38 45 I want to learn to understand other people better than 
I do now. 
Negative Correlations 
99 —.63 I want to earn more money than the average person. 
36 —.60 I want a college education so that I can earn a living 
more easily. 
95 —.59 I want to go to college because attendance at college 
makes it easier to get a job. 

_ 72 —.59 I want a college education so that I can lead an easy life. 
75 —.55 I want to avoid the hard life uneducated people live. 
55 —.54 I want the respect that people give to a college educated 

person. 
58 —.54 I want to appear educated to others. 
2 — 49 I think that going to college is the “thing to do.” 
4 — AT I want to go to college because my parents have always 
wanted me to go. 
105 —.47 I want to be able to get a job that will make me im- 
portant. 
57 — .45 I want to prepare myself for a better paying job than 
I could get without going to college. 
103 — 44 I want to reach a higher place in society than my 
parents have. 
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ture of an individual who gives high priority to intellectual growth 
and at the same time insists that considerations of financial and so- 
cial gain are of low importance to him. Factor II identifies a group 
of individuals who are greatly influenced by parental and social 
expectations and who give low priority to intellectual matters 
which were most important to Ss on Factor I. This procedure was 
followed in the identification and interpretation of subsequent fac- 


- tors. 
A brief description of each factor, which includes the items 
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TABLE 2 


Items Correlating with Factor II, Desire to Satisfy Parental 
` and Social Demands and To Have Future Security 
with Denial of Intellectual Motivation 


Item Correlation Content 
Positive Correlations 
44 .57 I want to go to college because my parents have always 
wanted me to go. 
104 50 I want to go to college because my parents think I’d be 
wasting a good opportunity if I didn’t. 
88 AT I want to go to college because the people who have in- 


fluence on me think it is the proper thing to do. 


95 45 I want to go to college because attendance at college 
makes it easier to get a job. 

36 42 I want a college education so that I can earn a living 
more easily. 

2 42 I think that going to college is the “thing to do.” 

22 A2 I want a college education because it is very important 
today. 

35 40 I want a college education because it will be a require- 
ment for most jobs in the future. 

41 38 I want to be able to provide financial aid for my parents 
in their old age. 

91 36 I want to go to college so that I will have a better 
chance for future happiness, 

Negative Correlations 

37 —.55 I want to get the answers to many questions. 

83 50 I want to specialize in one particular field in which I 
am very interested. 

1 49 I want to learn for the sake of learning. 

43 45 I want to further my education. 

39 —.44 I want to meet people who have intellectual and cultural 
interests. 

15 — 43 I want to satisfy my serious intellectual curiosity. 

100 — 42 I want to associate with nationally known scholars. 

76 —.40 I want to associate with college professors. 

[p - es I want to find out about certain kinds of knowledge. 


I want to go so that I can fulfill my purpose in life. 


with highest positive and negative correlations, is given in Table 3. 
Both the analysis and replication are presented with an indication 
of the factors which we feel are confirmed. Although the rotation 
was an orthogonal solution, a number of factors appear to be simi- 
lar. In most instances however the similarity is reduced by con- 
sidering the motives which those high on the factor deny. 

The matching of factors from the replication is similarly a mat- 
ter of judgment. Factors from the two analyses are paired when 
they have many items in common or when the implications of the 
items appear to be identical. In this analysis, Factors I, II, V, VII, 
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TABLE 3 


Factors from Original Analysis and from Replication with Items 
Showing Highest Positive and Negative Correlations 


Original Analysis Replication 
Factor Name Factor ~ Name 
I Desire for intellectual devel- I-R Same as factor I 
opment with denial of interest 8. I want to develop my 
in financial and social gain. abilities further. (+.61) 

34. I want to broaden my 57. I want to prepare my- 
overall viewpoint. (+.59) self for a better paying job 

99. I want to earn more than I could get without 
money than the average going to college. (—.43) 
person. (—.63) 

II Desire to satisfy parentaland II-R Same as Factor Vv 
social demands and to have 75. I want to avoid the 
future security with denial hard life uneducated people 
of intellectual motivation. live. (+.66) 

44, I want to go to college 47. I want to gain an 
because my parents have ` understanding of the intel- 
always wanted me to go. lectual and cultural heritage 
(+.57) of mankind, (—.50) 

37. I want to get the 
answers to many questions. 

(-.55) 

III Desire for vocational prep- II-R Same as Factor I 
aration and service with 90. I want to learn more 
denial of interest in personal about living. (+.55) 
development and pleasure. 54. I want to go to college 

82. I want a college degree so that I will have security 
because it is a necessity for in the future. (—.60) 
the kind of work I want to 
do. (+.53) 

30. I want to share in the 
fun that is part of college 
life. (—.54) 

IV Desire for academic develop- IV-R Desire for personal devel- 
opment with denial of inter- opment with denial of intel- 
est in increasing social status. lectual interests and need to 

16. I want to go to college achieve. 
because it is a challenge. 53. I want to learn how 
(+.46) to get along better with 

42. I want to be able to be other people. (+.58) 

a leader in civic affairs. 9. I want to go to the top 
(-.45) in my field. (—.50) 
V Desire for economic gains V-R Same as Factor IL 


with denial of intellectual 
interests. 

54. I want to go to college 
so that I will have security 
in the future. (+.49) 

11. I want to learn about 


4. I want to go to college 
because my parents have 
always wanted me to go. 
(+.54) 

48. I want to marry & 
college graduate. (—.45) 
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TABLE 3—Continued 
Original Analysis Replication 
VI Desire for economic and VIER Same as Factor VII 
social gains with denial of 17. I want to go so that 
j= interest in personal develop- I can fulfill my purpose in 
; ment. life. (+.48) 

99. I want to earn more 2. I think that going to 
money than the average college is the “thing to do.” 
person. (+.45) (—.42) 

26. I want a college edu- 
cation so that I will be a 
better husband. (—.44) 

VII Desire for leadership and VII-R Desire to improve social life 
service to society with denial with denial of intellectual 
of interest in social status. interests. 

33. I want to get a college 48. I want to marry & 
education so that I will be a r college graduate. (+.53) 
better than average citizen. 1. I want to learn for 
(+.40) the sake of learning. (—.60) 

46. I want to develop my 
social skills. (—.48) 

VIII Desire for prominence ‘YVIII-R Same as Factor IX 
through service with denial 67. I want to lead my 
of personal interests. own life. (+.61) 

65. I want to take part in 65. I want to take part in 
extra-curricular activities. extra-curricular 
(+.38) activities, (—.40) 

27. I want to associate 
with the kind of person I’d 
like to marry. (—.48) 

Ix Desire for independence with IX-R Desire to serve society with 
denial of social interests. denial of interest in vocational 
_ 67. I want to lead my own preparation and social life. 
life. (+.68) 32. I want an education so 

86. I want to associate that I can be more of a credit 
with the kind of person I io my community, state and 
would like to be. (—.32) nation. (+.39) 

93. I want to meet people 
whose goals are similar to 
mine. (—.41) 

X Desire for materialistic gains X-R Same as VIII 
with denial of social and - 42. I want to be able to 
intellectual interests. be a leader in civic affairs. 

54. I want to go to college (+.47) 
so that I will have security 77. I want to be able to 


in the future. (+.56) 
48. I want to marry a 
college graduate. (—.46) 


afford a family. (—.44) 
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VIII, and IX are considered confirmed by the second study. The 
positive sides of these six factors have been identified by more 
traditional methods of interviews and surveys. In earlier studies 
they have been variously named. Factor I is the rather obvious in- 
tellectual motive for college attendance. Factor II is attendance re- 
sulting from a conformity to family and social pressures. Financial 
security is expressed in Factor V. Leadership and citizenship are 
prominent motives for those on Factor VII. Social prestige and 
prominence are important to those on Factor VIII while those on 
Factor IX go to college in order to achieve independence. Factor 
III, which emphasized vocational purposes and on which 25 Ss 
showed high loadings, was not confirmed in the second analysis. 

The unusual aspects of our results are the identification of lowly 
valued goals in association with the sets of important goals which 
make up each factor. Thus, the downgrading of social and financial 
benefits appeared to be part ‘of the picture where intellectual in- 
terests are paramount. Those who are characterized by con- 
formity tend to minimize intellectual purposes. Those who em- 
phasize vocational purposes minimize the personal pleasures of 
college life. The purposes which are given consistently low values 
for a given group are probably as important for understanding the 
behavior of the members of that group as are their highly valued 
goals. The values which members of one group deny may be 
equally as strongly asserted by members of another factor. 

In any given academic setting the primary set of values en- 
dorsed by the faculty may or may not coincide with those of in- 
dividual students. A given student may find that he gives low pri- 
ority to the purposes endorsed by the institution and as a result 
may not perform as well as he would where his values were 
shared. 

The results of our analyses are to an unknown degree specific 
to males from one university and the pool of items used. It will be 
necessary to have data from other samples of subjects and items 
before great confidence can be placed in a specific factorial solu- 
tion, The problems of the number of factors and of orthogonal or 
oblique solutions also remain. It will also be necessary to determine 
whether predicting performance of these more or less homogene- 
ous subgroups will involve changes in beta weights and/or predic- 
tors for each group. 
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This research has been concerned with patterns of motivation 
for college attendance. A 105 item Q-sort of statements for at- 
tending college was compiled and given to 200 male undergradu- 
ates, The sample was divided and two inverse factor analyses were 
done. Six of the ten factors described in the first analysis were con- 
firmed by the second. Implications for prediction and for counsel- 
ing were considered. 
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PERSONALITY PATTERNS IN PRE-ADOLESCENT 
DELINQUENT BOYS"? : 


HERBERT C. QUAY 
University of Illinois 


In a series of prior studies we have attempted to determine the 
principal dimensions of ty associated with juvenile de- 
linquency in samples of institutionalized adolescents. The factorial 
analysis of data obtained from case histories (Quay, 1964b), ques- 
tionnaire responses (Peterson, Quay and Cameron, 1959; Peterson, 
Quay and Tiffany, 1961; Quay and Peterson, 1964), and ratings of 

behavior (Quay, 19640) has revealed a remarkable con- 
ceptual similarity in factor structure, Generally, the majority of 
variance is accounted for by three factors which we have labeled 
psychopathic-unsocialized, neurotic-disturbed, and subcultural- 
socialized. A fourth factor, usually accounting for less than 10 per- 
cent of the variance, has also emerged on occasion and we have 


considerable similarity to that suggested by other studies of prob- 
lem behavior in nondelinquent as well as delinquent children, 


(Hewitt and Jenkins, 1946; Jenkins and Glickman, 1947; Peterson, 


1961; Quay and Quay, 1965). 
Since the development of a system of statistically covarying and 


1 This research was by ganta (M5027 and Eaton 
the Taiga) eee SS 8. Department of Health, Education 
d eituthor wishes to express his appreciation to Mr. Harry R- 
Chief, Division of Institu California Department of the Youth 
and Mr. Noel G. Bonelli his staff of the Fricot Ranch School for 
generous cooperation in this study. 
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conceptually coherent personality dimensions relevant to delin- 
quency has important implications for studies of both etiology and 
remediation the present study sought to extend the earlier work to 
a sample of younger delinquent boys. 


Method 
Subjects $x 


Ss were 122 males institutionalized in a comparatively small state 
institution with a specialized program for younger delinquents. 
The mean age of the sample was 12.27 years with an SD of 1.36. Ss 
studied represented a sizeable proportion of the entire institution 
population and can certainly be considered representative of the 
total population. The institution itself handles almost all of the 
serious younger delinquents in the state in which it is located. 


” Procedure 
Behavior Ratings j 
Ratings of behavior were obtained from two sources: classroom 
teachers and cottage supervisors. The rating scale, containing 58 
items descriptive of problem behavior, had been used in an earlier 
study (Quay, 1964a) and ‘had been originally developed by Peter- 
. gon (1961), While raters were asked to differentiate “mild” from 
“severe” problems, in the statistical analysis ratings were simply 
dichotomized on a “present” or “absent” basis. A total of 221 rat- 
ings were analyzed; the duplicate ratings on 99 of the Ss allowed 
an assessment of the reliability of the factor scores obtained. Items 
which were noted in less than 10 percent of the cases were not 
further analyzed, The remaining 49 items were intercorrelated by 
means of Phi-coefficients, subjected to a principal axis factor analy- 
sis and rotated to the varimax criterion (Kaiser, 1958). Squared 
multiple correlations were used as communality estimates. On the 
basis of the prior research an arbitrary decision was made to rotate 
only three factors, Inspection indicated that among the three addi- 
tional factors extracted but not rotated there were no loadings 
greater than .50 and only two greater than .40. In previous work we 
have routinely not rotated any factor not having at least one load- 
ing of .40. 
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Case History Analysis 


The analysis of the case histories was accomplished by social 
workers who were familiar with the records of the Ss. Ratings of 
“present” or “absent” were made on a checklist containing 36 
items. Any item noted in less than 10 percent of the cases was 
dropped; the remaining items were analyzed in exactly the same 
way as were the*behavior ratings. In this analysis there were only 
three loadings of .40 or greater among the three factors extracted 
but not rotated. s 

It should be noted that the reliability of the data contained in the 
case histories is of course unknown. Assessing of rater reliability of 
judgments from this data was also impractical within the limits of 
this study. However, as has been true inthe previous studies of his- 
tory ratings (Hewitt and Jenkins, 1946; Quay, 1964b) the results 
do not appear to be seriously attenuated by lack of i from 
either source. p” S 


Personality Questionnaire ~a * 


Ss were also administered a factorially derived questionnarie 
which had been designed to measure three personality factors from 
this domain previously found to be associated with delinquency 
(Quay and Peterson, 1964). In addition to considerable factorial 
purity for the subscales all of the items in the questionnaire statisti- 
cally differentiate delinquents from nondelinquents. Ss were ad-~ 
ministered this inventory in classroom groups; in the lower grades 
the items were read aloud to Ss by Æ. Factor scores were obtained 
for each S and these scores were subsequently related to factor 
scores obtained from the other media. 


Results 
Behavior Ratings 


The three factors rotated accounted for 65 percent of the com- 
mon factor variance. Table 1 presents the rotated factor loadings in 
company with loadings for the same variables from two other 
samples. The first factor clearly has to do with overt aggression and 
hostility coupled with irritability, defiance and an absence of con- 
cern for others, In keeping with prior work we have labeled this’ 


7. 
5. 


47, 


30. 


37. 
32. 
41. 
29. 
13. 
20. 
44, 
40. 


21. 
39. 
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factor psychopathic-unsocialized. The second factor is obviously a 
dimension of anxiety, depression, inferiority and withdrawal, In 
accord with previous work this factor has been labeled neurotic- 
disturbed. Factor III is less clearly interpretable and accounts for 
only 9 percent of the common factor variance. The high loadings 
on shortness of attention, easily flustered, shyness, lack of interest, 
laziness and daydreaming suggest an inadequacy-immaturity reac- 
tion and the factor has been so labeled. 


TABLE 1 


Rotated Factor Loadings for the Ratings of Problem Behavior with Loadings 
from Previous Studies Presented for Comparison 


Adolescent Eighth Grade 
Present Study. Delinquents Students 
Variables I P 1 ORENA 1" ORAN I I T re TY 
Boisteriousness, " y 
rowdiness. -69 .00 .08 .48 “11 —.14 18 -60 —.03 .15 
Disruptiveness, 


tendency to annoy ‘ 

and bother others .67 .00 .15 .47 77 —.06 11 .70 —.12 21 
Irritability; hot- 

tempered, easily 

aroused to anger. 65 .16 —.15 .47 -538 29.87 e 
Disobedience, 

difficulty in disci- 

plinary control, 64—.03 4.06 .41 7A 11 00 00 22 
Hyperactivity; 

“always on the go.” .60 —.05 .09 .37 -387 .03 .53 -54 .08 —.01 
Uncooperativeness 

in group situations. .59 —.04 .13 .37 74.15 OL .36 —.07 .25 
Impertinence * 

sauciness, rE RAO AE o AE o Aa 288/583 11 R 
Tension, inability 

to relax. .57 .28 .09 41 

Jealousy over at- 

tention paid other 
children. 56 28 —. 
Fighting .55 .02 —. 
Profane language, 

swearing, cursing. .55 .06 —.03 .31 

Negativism, tend- 

ency to do the 

opposite of what 

is requested. -53 04 .27 .36 

Temper tantrums. .52 .22 —.05 .35 .44 .25 44 
Destructiveness 

in regard to his 

own and/or others 

property. -51  .06 18.29 
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17. 


46. 


14. 


Variables 


Present Study 


Ian 


Adolescent 
Delinquents 


III 


Eighth Grade 
Students 


TEN I 


Inattentiveness to 
what others say. 


. Attention-seeking, 


“show-off” behav- 
jor. 


. Distractibility. 
. Restlessness, in- 


ability to sit still. 


. Feelings of infer- 


jority. 


. Easily flustered 


and confused. 


. Anxiety, chronic 


general fearful- 
ness. 


. Shyness, bashful- 


ness. 


. Self-conscious- 


ness; easily em- 
barrassed. 


. Lack of self- 


confidence, 


. Depression, 


chronic sadness. 


. Hypersensitivity; 


feelings easily 
hurt. 


. Social withdrawal, 


preference for 
solitary activities. 


. Clumsiness, awk- 


wardness, poor 
muscular coordin- 
ation. 
Nervousness, 
jitteriness, jumpi- 
ness; easily 
startled. 


- Doesn’t know how 


to have fun; be- 
haves like a little 
adult. 

Prefers to play 
with younger 
children. 


. Crying over minor 


annoyances and 


84 


01 


39 


12 


„17 


.52 


AL 


40 


-39 


+25 


07 


27 


+26 


17 


31 


36 


+23 


-16 


19 


19.55 


K381 51l 


22 42 


.23 60 


.52 


„04 


18 
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TABLE 1—Continued 


Adolescent Eighth Grade 
Present Study Delinquents Students 

Variables I w- pr as I TE | AEE I SL Ik 
25, Laziness in school 

and in perform- 

ance of other 

tasks. -24 .09 .66 .49 -55 .20 .00 .22 —.15 .89 
28. Masturbation. —.0l .27  .62 .46 10), 17i :68 
42, Sluggishness, 

lethargy —.16 ` .29 .58 .45 22 62 —.05 EA y ADA i ERE» 
43. Drowsiness. —.07 .25 .52 .34 
15. Short attention 

span. 77 A NL O. A ities b E- 44 11.56 


9. Preoccupation; 

tin a world of 

his own.” —.21 .18 .48 31 14 .60 .28 —.04 .21 62 
19. Lack of interest 

in environment, 


generally “bored” 

attitude. —.03 .29 .47 .3l .49 .49 —.02 —.02 —.02 .48 
27. Excessive day- 

dreaming. .41 —.07 .50 .42 .08 .70 .26 18.20. .87 
12. Dislike for school. .B1 17 «40 | 1.28 
22. Reticence, se- 

cretiveness. —.04 .14 .25 .08 —.23 .18 .08 
23. Truancy from 

school. 04 —.03 .05 .00 
33. Aloofness, social 

reserve, —.30 .20 .21 17 —.0l .20° 02 —.32 .22 .20 


34. Passivity, suggest- 
ibility; easily led 


by others. 51D 28 MRS AEE 47 62130 .13 .14 38 
36. Stuttering. .18. 12 16 .07 
45. Prefers to play 

with older child- 

ren. +03 .00 —.03 .00 
48, Enuresis, bed- 

wetting. -10 .08 .08 .02 


49. Specific fears, 
e.g., of dogs, of 


the dark. 08 19 09.05 
% common 
variance 35 2i 9 49 17 7 Bl 28i 12 


* Where no loadings appear for the earlier studies the particular variable was not analyzed due to infrequency 
in the sample. 
By inspection the comparability between the results of this study 
and those of previous studies on the samples of both delinquents 
(Quay, 1964a) and public school students (Quay and Quay, 1965) 
is quite good. This improvision is well substantiated, at least for the 
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first two factors, by the pattern of Tucker coefficients of factor 
similarity? which may be found in Table 2. 


TABLE 2 


Coefficients of Factor Similarity for Loadings in Table 1. 
“Validity” Values Underlined. 


Adolescent Eighth Grade 
Delinquents Students 
Factors I IL II I I Til 
Present Study 
I. Unsocialized-Psychopathic  .87 27 48 93 —.06 45 
II. Disturbed-Neurotic 23 «8668.09 72.27 
III. Inadequate-Immature .62 .62 .49 .26 .29 .89 


Case History Analysis 


Four factors accounted for 66 percent of the common factor 
variance. Rotated factor loadings may be found in Table 3 in com- 
pany with loadings for the same variables obtained in the earlier 
study of institutionalized adolescents (Quay, 1964b). Factor I 
seems a clear representation of a dimension of aggression and overt, 


TABLE 3 
Rotated Factor Matrix for Case History Variables* 


h? 


84 


36 


17 
33 © 
33 


19 
25 


03 
31 


Present Study Adolescent Delinquents ` 
Factor Factor 
Variable I TD) PEL Lay tet I EU ees 4, LV; 

Assaultive 59 -05 -—05 —03 35 54 14 —16 —03 
Has bad companions 00 73 08 —09 55 06 59 —02 —08 
Seclusive, stays to 

himself —34 -28 38 —03 34 -14 —38 07 08 
Initiates fights 68 —03 —11 —07 48 404 58 —16 —13 
Shy —45 -11 58 10 56 -il -33 46 04 
Cruel 43 —11 —02 —12 21 
Engages in co-oper- 

tive stealing —10 63 03 —36 54 —06 15 —02 41 
Apathetic, emotionless —09 —02 42 = 07 19 
Quarrelsome 59 09 03 09 37 43 —09 —12 —20 
Loses interest quickly 10 -08 27 —17 12 —05 —05 —08 13 
Defies authority 46 12 -04 32 33 47 14 -26 06 
Engages in furtive 

stealing 07 -08 13 48 21 18 —25 —15 02 


3 Ledyard R Tucker, personal communication. This coefficient is obtained 
by dividing the sum of the cross-products of the loadings by the square root 
of the product of the two sums of squares. It is essentially a correlation coef- 
ficient, uncorrected for origin. 
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TABLE 3—Continued 


Present Study Adolescent Delinquents 
Factor Factor 
Variable I Ty ON? he I i 05 0 a a) 
Worries 01 12 47 -03 24 -07 -13 67 04 47 
Engages in malicious 
mischief 34 12 37 -04 27 01 00 —21 06 05 
Habitually truant from 
school —17 -07 -23 05 09 17 2 04 11 09 
Sensitive -0 00 34 -04 12 13 -04 70 —0l1 5l 
Unable to cope with a 
complex world 14 -19 49 47 52 04 02 06 55 30 
Timid —43 —05 56 06 50 -09 -17 59 15 41 
Has inadequate guilt 
feelings 25 14 -04 44 28 144-10 -27 -14 12 
Habitually truant 
from home —06 —13 -25 -01 08 -277 27 -21 42 37 
Submissive -382 -Ol 384 08 22 
Stays out late at nights —01 52 —13 05 29 —05 48 —26 25 36 
Irritable 48 00 20 -15 29 61 03 10 00 32 
Accepted by a delin- 
quent subgroup 09 73 -01 08 55 -15 63 09 —32 53 
Lonesome -13 -32 34 21 28 
Verbally aggressive, 
impudient 61 22 -09 -09 44 56 -04 —19 00 35 
Strong allegiance to 
selected peers 08 61 22 -05 43 13 86 —07 -29 24 
Incompetent, 
immature 05 -18 29 2 18 08 —20 02 36 17 
Obscene, uses foul 
language 49 14 OL 12 27 
Feels persecuted, be- 
lieves others unfair 29 00 11 -02 10 49 16 OOF 15 29 
ahaa gil 2 03 1 -02 
vior - i - 07 -07 -23 42 0 23 
Callous, little concern 
for others 22 18 -02 -38 23 16 16 —26 -06 12 
Unable to profit by 
either praise or i 
punishment 20 —12 14 -37 21 81 -28 —31 -03 27 
Suspicious, trusts no 
one 24 -10 -07 15 10 00 -12 -19 -12 06 
Has engaged in sex 9 
delinquencies —09 16 -05 26 10 
% common variance 27 WwW 4 8 26 15 10 9 
* decimals omitted 


unbridled hostility and in keeping with prior studies has been 
labeled psychopathic-unsocialized. The second factor reflects a 
syndrome of gang activities and as in the past has been labeled un- 
socialized-subcultural. Factor III contains elements of withdrawal, 
submissiveness and anxiety and appears to warrant the application 
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of the disturbed-neurotic label. The fourth factor is again less well 
defined but its components suggest the inadequacy-immaturity 
label applied in the earlier studies, Coefficients of factor similarity 
have been calculated and these may be found in Table 4. These 
coefficients indicate a high degree of similarity in the structure ob- 


tained from the two samples for all but factor IV. 


TABLE 4 


Coefficients of Factor Similarity for Loadings in Table 8. 
“Validity” Values Underlined. 


Present Study Adolescent Delinquents 
Factor 
I Iv 
I. Psychopathic-Unsocialized 73 380 —.46 —.17 
II. Socialized-Subcultural -08 ¿85 —.10 —.20 
III. Neurotic-Disturbed 03 oA T7 38 
IV. Inadequacy-Immaturity OL -.18 —.01 4 


Factor Scores 


Table 5 presents the means and standard deviations for the factor 
scores from the three domains. These values are comparable to 
those obtained in the earlier researches on institutionalized adoles- 


cents, 
TABLE 5 
Means and Standard Deviations for Factor Scores. 
Domain and Factor Mean 8D 
Questionnaire 
Psychopathic-Unsocialized 12,22 8.10 
Neurotic-Disturbed 15.80 5.52 
Socialized-Subcultural 18.01 3.27 
Behavior Rating (Teacher) 
Psychopathic-Unsocialized 7.09 5.18 
Neurotic-Disturbed 5.59 4.05 
Inadequate-Immature 4,42 3.27 
Behavior Rating (Cottage) 
Psychopathic-Unsocialized 8.15 5.07 
Neurotic-Disturbed 6.31 3.31 
Inadequate-Immature 4.15 2.60 
Case History 
Psychopathic-Unsocialized 1.96 2.49 
Neurotic-Disturbed 1.50 1.48 
Inadequate-Immature 1.23 1.22 
Socialized-Subcultural 1.43 1.62 
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As noted above 99 of the Ss had been rated by both teachers and 
cottage supervisors. Correlations between these two sets of ratings 
were quite low; .38 for Factor I, .20 for Factor II and .30 for Factor 
III. These reliabilities are even poorer than those found previously 
(Quay, 1964a); nevertheless while there is obviously both situa- 
tional and rater variance (in addition to considerable restriction of 
range) the raters are using a common dimensional system. 


Factor Equivalence Across Domains 


In order for a factor analytically derived descriptive system'to be 
most meaningful some degree of equivalence of conceptually simi- 
lar factors evolving from different domains ought to be empirically 
established. The establishment of such linkage has been a point of 
contention and a serious problem for factor analytic research on 
personality (Becker, 1960; 1961; Cattell, 1961a; 1961b). 

We have approached the establishment of empirical validity for 
cross-domain comparisons by obtaining factor scores for each S 
on all factors, intercorrelating these scores, and presenting the re- 
sulting intercorrelations (Table 6) in a multitrait-multimethod ma- 
trix (Campbell and Fiske, 1959). 

On the whole the validities are poor and the within method cor- 
relations are frequently higher for different traits than the between 


TABLE 6 


Correlations among Factor Scores for Questionnaire, Behavior Ratings 
and Case History Data “Validity” Values Underlined. 


K i Domain 
Questionnaire Behavior Ratings* History 
Factors N P S N P I SP IFS 

Questionnaire P -30 

S .49 .02 

N u 2 03 
Ratings R 08 .16 .03 22 

I 04 31 —.06 57 37 

N =.07 —.01 .00 .19 —.26 07 
History P +15 —.08 .18 —.03 .33 —.09—.41 

I —.03 —.03 .08 0r —114 —.01 35.00 

S —.17 —.04 -01 —.17 —.04 —.20 -05 .05 —.09 


* Based on the averages of values obtained from teachers and cottage supervisors. 
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correlations for the same traits. It is clear that the test factors have 
little empirical validity for the differential prediction of any of the 
factors from the other domains. However, for both psychopathy 
and neuroticism there is some suggestion of cross-domain matching 
between the history analysis and the behavior ratings; here the cor- 
relations between cross-domain factors are higher for the same trait 
than for the others. 
Discussion 

A*number of conclusions seem appropriate. When one considers 
data from within either the case history record or behavior rating 
domains the structure similarity across varying samples is striking. 
This is true not only for other samples of delinquents but for 
public school children as well. Thus, it appears that the primary 
dimensions of problem behavior as manifest within each domain are 
common to all children. 

When cross-domain relationships are considered convergence is 
much less clear. One can speculate about the causes of this lack of 
convergence and this has been done at length (Cattell, 1961a). In 
this instance it is clear that the institution setting (from which come 
the behavioral ratings) and the prior life setting (from which come 
the history data) are situational contexts separated temporally as 
well as spatially. Further there are reductions in the magnitude of 
the relationships resulting from the known low reliability of the 
behavior ratings; poor reliability likely characterizes the case his- 
tory data also. 

The self reports about behavior and attitudes required by the test 
are certainly not highly related to the judgments about concep- 
tually similar behavior and attitudinal dimensions provided by 
others, Here again both situational and psychometric factors are at 
work. In addition the problem of defensiveness and other response 
biases on the part of the Ss must be considered. 

In view of the ample opportunities of operation for both situa- 
tional and more purely psychometric influences in the cross- 
domain comparisons we are inclined to feel that the present results 
may be all that can be expected and even the limited magnitude of 
the cross-domain relationships argue for the presence of basic trait 
dimensions which are observable in all three domains. On the basis 
of this research and the earlier studies these dimensions now appear 
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to be well enough established to suggest that their origin and main- 
tainence are of considerable importance in the understanding of 
juvenile delinquency. 
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Asour two decades ago Cattell (1944) proposed a broad logical 
distinction between the behavior persons exhibit under instructions 
to answer questions about themselves and that revealed when re- 
quested to find the “best” answer or otherwise describe or classify 
stimuli. He suggested that responses gathered in the first- 
mentioned manner be termed Introspective, self-estimate, or Q- 
data, and that behavior observed in the latter way be called Objec- 
tive-test or T-data (both to be contrasted with ratings by others or 
L-data). More recently Jackson and Messick (1958) and Torger- 
son (1960) have discussed essentially the same distinction under 
the heading of “responses” and “judgments,” while Coombs (1953) 
drew a similar comparison in his classification of Type I and Type 
II models. This basic distinction is currently at the crux of much of 
the controversy over the place which the concept (or concepts) 
of social desirability should occupy in personality or general psy- 
chological theory. 

Several studies (Cattell, 1957; Edwards, 1957 ; Heilbrun and 
Goodstein, 1961; Loehlin, 1961; Messick, 1960; Rozynko, 1959) 
have indicated that the ways in which persons answer questions 
about themselves are related to the ways in which they judge (per- 
haps value) the behavior they are instructed to attribute to them- 
selves, Specifically, it has been shown (Edwards, 1957) that, if a 
respondent judges certain behavior to be “socially desirable,” he is 
more likely to ascribe it to himself than if he considers it to be 
“andesirable.” But it is now quite apparent from the work of 
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Loehlin (1961), Messick (1960), Morris and Jones (1955), Osgood, 
Suci and Tannenbaum (1957) and others, that there are note- 
worthy differences in the patterns of behavior (beliefs, etc.) peo- 
ple regard as desirable. Messick, for example, using a sample of 42 
stimuli, found that nine factors were needed to account for the in- 
tercorrelations between judgments about the desirability of be- 
havior described in these rather typical self-report items. Some of 
this work, notably that of Buss (1959), Jones and Thurstone 
(1955), Loehlin (1961), and Osgood (1957) suggests that a few of 
these dimensions of “judgment” may be closely related to differ- 
ences in language usage or in the actual concepts which the language 
terms are used to “tag.” 

It has long been apparent (Cattell, 1957), however, that there are 
several consistent and distinct patterns of behavior which people 
are likely to accept or reject in self-ascription. Edwards (1957) has 
documented a case for the contention that a substantial propor- 
tion of the covariation among such self-reports can be accounted 
for by a general set of cultural standards in relation to which most 
personal and social behavior can be assessed as desirable or unde- 
sirable. Granting that this general dimension represents a poten- 
tially useful concept, it must be noted that it does not account for 
all of the reliable variance of lower order factors (Horn, 1963). 

In sum, whether these phenomena are approached through the 

* methodology of objective tasks, or via subjective self-assessments, 
- the evidence suggests that (1) there are probably several note- 
worthy dimensions of individual differences in conceptions of 
what are and what are not socially desirable characteristics of peo- 
ple, (2) there are probably also several significant dimensions of in- 
dividual differences in self-evaluations, (3) lawful relationships be- 
tween the dimensions of (1) and (2) probably exist, (4) there is a 
definite need to establish more firmly the salient dimensions in 
both (1) and (2), and to specify precisely the correspondence be- 
tween these dimensions. The present study represents one attempt 
to deal with these questions, particularly with (4). 


Method 


Seven putative measures of a social desirability attribute of indi- 
vidual differences were studied to determine first, if the hypothe- 
sis that all can be regarded as operationally representing a single 
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concept could be retained and, second, if this hypothesis had to be 
rejected, the number and nature of dimensions reliably assessed by 
these measures. 


Subjects 


A complete set of responses for all measures was obtained from 
each of 50 high school seniors. 


Measures 


The scales here studied, all purported to be measures of a social 
desirability “set,” are classified in Table 1 as either Q-data or 
T-data devices, following as nearly as possible the criteria set down 
by Cattell (1958) and Scheier (1958). The scales were presented to 
all Ss in the orders specified in a 7 by 7 Latin Square. The ra- 
tionale for use of each scale is discussed below. 


TABLE 1 
SD Measures 


T-Data Q-Data 


Desoto “Social Approval” (DSA) Bills Perceived Self (BPS) 
Borislaw ‘Perfect Person” (BPP) Desoto Well Being (DWB) 
Bills Ideal Self (BIS) Edwards Social 
Desirability (ESD) 
Marlowe-Crowne Social 
Desirability (MCSD) r 


The Edwards “scale” contains items selected because the be- 
havior implied by the keyed response was consistently rated as 
highly desirable by the judges employed in the test construction 
studies (Edwards, 1957). As here used, the scale requires the sub- 
ject to endorse the item statement for the behavior in question rel- 
ative to himself. 

Crowne and Marlowe (1960) argued that the content of the Ed- 
wards scale, based on items drawn from the MMPI, is overly re- 
srticted to questions about symptoms of mental illness and is, there- 
fore, primarily a measure of a desire to avoid labeling oneself as 
“sick.” Using methods similar to those employed by Edwards, they 
developed a self-rating instrument, the Marlowe-Crowne SD scale 
(MCSD), containing few items which involve pathological content. 

The Bills Index of Adjustment and Values (IAV) (Bills, 1958) 
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was originally designed to provide three measures, the two em- 
ployed here and the Perceived Self Scale which instructs the sub- 
ject to rate himself as he in fact is, while the other, called the Ideal 
Self Scale (BIS), asks the respondent to make his ratings in accord- 
ance with the kind of person he would ideally like to be. A judg- 
ment scale for desirability of these items was found (Cowen and 
Tongas, 1959; Spilka, 1961) to be nearly a perfect linear trans- 
formation of that based on the Edwards Social Desirability Scaling 
Item Procedure. The original scales were, therefore, interpreted as 
largely measuring social desirability. 

The Borislaw (1958) “Perfect-Person” measure (BPP) derives 
from the theory and procedure used by Ruch (1942) that a social 
desirability “set” will be indicated by the responses of subjects in- 
structed to answer questions in the way they believe the “perfect 
person” would answer. Since this requires the Ss to rate an external 
“object,” the measure is seemingly objective by Cattell’s definition. 

A similar rationale is given for the “Social-Approval” measure 
(DSA) developed by DeSoto, Kuethe, and Bosley (1959). Here the 
subject is instructed to respond in a way that would, if the re- 
sponse were examined by others, gain him social approval. These 
authors also provided a scale of “well being” (DWB) thought to be 
confounded by social desirability and here classified as a Q-data 
measure. 


Procedures and Analysis 


One of the best developed linear models for examining the first 
hypothesis stated above is still that created by Spearman (1904) 
and his students (Spearman, 1927). This embodies a stringent set 
of criteria, but allows one to retain a hypothesis with considerable 
confidence if the requirements of the model are satisfied. In essence 
it demands that scales measure one and only one attribute in com- 
mon. The total variance of the measures need not be accounted for 
by this one factor, since the model explicitly allows that measures 


1It should be noted, however, that when the subjects in Spilka’s (1961) 
study were instructed to endorse the items to describe themselves or their 
ideal selves, the measures correlated 36 and —.09 respectively with the self- 
ratings of ESD. These results thus rather clearly support the position taken 
by Thurstone and Chave (1929) over 30 years ago to the effect that there is 
no necessary relationship between scales derived by judgmental approaches 
and scales of endorsement. 
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may be unreliable or may reliably measure attributes not also 
measured by other scales in a battery, but all common variance, as 
represented by the intercorrelations, will be exhausted by a single 
dimension. 

Examination of the intercorrelations among our measures of so- 
cial desirability (Table 2) reveals that the requirements of the 
Spearman model were not satisfied by these data, although positive 
manifold, indicating the presence of one (but not only one) general 
factor in a hierarchical solution (Schmid and Leiman, 1957) is 
definitely suggested. 


TABLE 2 


SD Intercorrelation Matriz (Below Diagonal) Residuals After Extraction 
of 3 Factors (In and Above Diagonal), and Spearman “g” Loadings 


DSA BPP BIS BPS DWB ESD MCSD 


DSA —001* —000 001 —001 —001 010 —004 
BPP 684 —005 026 —002 004 —009 017 
BIS 686 584 002 000 —003 026 —029 
BPS 684 567 652 —003 024 —029 024 
DWB 516 596 336 407 | 000 —002 —007 
ESD 118 144 040 141 143 031 —018 
MCSD 033 095 —072 146 067 412 | 007 
ttg! 

loading 455 519 252 650 331 226 022 


* Correlations and residuals are here recorded to only three places (with decimal points omitted), 
although actual calculations carried ten places throughout. 

Since the hypothesis of one factor could not be retained, a mul- 
tiple-factor hypothesis was explored by use of the methods of prin- 
cipal axes factor extractions with reduced communality estimates 
(Harman, 1960). Unities were inserted in the principal diagonal of 
the matrix of correlations and all principal factors were extracted. 
The latent root of the fourth factor proved to be less than 1.0, 
while that of the third factor was 1.03. Hence, following the 
proofs and arguments of Guttman, Kaiser, and Tryon (Harman, 
1960), it was estimated that three common factors could be re- 
liably determined. Five centroid iterations were carried out to 
estimate the reduced communalities (entered in the fourth column 
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of Table 3), these were inserted in the diagonals of the correlation 
matrix and three principal axes factors were computed. The residu- 
als, given in and above the principal diagonal in Table 2, were all 
essentially zero at this point, the largest being 0.031. The results 
from this analysis are given in Table 3. 


TABLE 3 
Principle Axes Factors and “Entered With’ h 

I 1 m ht 
DSA 859 —101 040 749* 
BPP 306 000  —199 683 
BIS 744 —225 213 623 
BPS 799 009 279 738 
DWB 682 055  —431 583 
ESD 197 620 029 455 
MCSD 116 654 083 455 

3.03 1.88 0.36 
% COMMON 70.1 20.4 8.4 


a ŘŘaIIamĖŐ—— mmm 
* These are the reduced communalities which resulted from five centroid iterations. They will 
differ slightly from the sums of squares across factors, 


As would be expected from an inspection of the intercorrela- 
tions, these results show that a major portion of the common vari- 
ance (and, indeed, the major portion of the reliable variance of the 
first five variables) is accounted for by the first factor and that the 
ESD and MCSD form a substantial doublet factor largely inde- 
pendent of the first. The third factor would probably not be iden- 
tified by inspection of the correlations alone. It is of borderline 
significance by Kaiser’s (1958) test, as noted above and hence will 
not be emphasized in our later discussions. 

The first factor involves all of the objective measures, suggest- 
ing high evaluation of self and a claimed sense of well-being. The 
self-evaluations made in the Edwards and Marlowe-Crowne de- 
vices, form a quite distinct factor, although low objective ratings 
of the “ideal self” also produce some variance. 

In an attempt to retain the hierarchical order of the dimensions, 
while improving their interpretability and increasing the likeli- 
hood of invariance, the factors were rotated to that approximation 
to simple structure which is given by Kaiser’s (1958) Varimax 
Criterion. These results are presented in Table 4. 

The first two factors are not much altered by this rotation, al- 


x 


BERNARD SPILKA, ET AL. 117 


TABLE 4 
Principal Azes Factors Following Varimaz Rotation 


I II II 
DSA 753 043 426 
BPP 564 105 600 
BIS 773 —007 217 
BPS 807 169 194 
DWB 284 104 702 
ESD 056 642 090 
MCSD 011 670 002 


though the well-being measure loads substantially less in the first 
and “ideal self” ratings sink into the hyperplane of the second, but 
the third factor is found to be defined by a stated sense of well- 
being, objective ratings of the “perfect person” and the attempt 
to reply in a way that would earn social approval. 


Discussion 


A substantial proportion of the reliable variance in all measures 
is involved in the three factors here obtained, but the Edwards and 
Marlowe-Crowne devices, particularly, are not fully defined in this 
space. Judging from the results of previous work (Horn, 1963), 
these two variables would enter prominently into a general anxiety 
factor, although it is now also well established (Edwards, 1963; Ed- 
wards, Diers, and Walker, 1962; Horn, 1964) that, if the sampling 
of variables is concentrated in the Q-data medium alone, the ESD 
and MCSD will fall on somewhat distinct factors. Thus, these two 
variables here define what would be regarded as a second-order 
factor in many analyses. 

From our point of view, the more interesting finding is the rela- 
tionship between the objective (i.e., T-data) and subjective (i.e., 
Q-data) rating tasks as represented by the first factor in the princi- 
pal axes or Spearman solution. This could be due to consistency in 
the use of scales (i.e., be an extremity or “in between” response set). 
More reasonable is the notion that these measures are keyed accord- 
ing to what, on the average, in the upper and middle classes of our 
society are regarded as socially desirable qualities, and that some 
people more nearly agree with this definition of the “good guys” 
than do others. This may be termed a social desirability “set”—but 
it must be noted that in our results this is largely independent of 
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the stereotype defined by the keys of the Edwards and Marlowe- 
Crowne devices. The latter fact may be due to the influence being 
observed in the recent studies of Edwards and Heathers (1962) and 
Edwards, Diers, and Walker (1962) in which very high correlations 
are demonstrated between first factor loadings on the MMPI and 
61 personality scales with ESD. This suggests that contrary to in- 
tentions, the Marlowe-Crowne, like the Edwards, contains a sub- 
stantial portion of variance due to pathological item content. With 
loadings as low as those here observed in Factor II, the alternative 
hypothesis that this factor indicates commonness between the non- 
pathological subcomponents of the Edwards and the Marlowe- 
Crowne devices cannot be rejected. 

Rather than using the now obviously ambiguous term “social de- 
sirability,” we would suggest at least a tentative identification of 
Factor I with the seemingly broader concept of self-sentiment dis- 
cussed by Cattell (1957) and more fully in the reference cited. 
Factor II would seem better considered within the context of the 
concept of anxiety, a term which is no more ambiguous than social 
desirability and one for which the relational fertility in general 
psychological theory is definitely greater. 

Since the third factor is of only borderline significance, we will 
not occupy space with a discussion of it here. 


Summary 


An analysis by factor analytic methods of the correlations among 
seven putative measures of a social desirability “response-set” re- 
vealed that (1) there is no single attribute of social desirability 
“set,” (2) although objective measures of what are and what are 
not desirable qualities of persons are related to similar self- 
evaluation measures and define what might tentatively be inter- 
preted as a self-sentiment factor, this dimension is largely inde- 
pendent of that defined by the two currently most popular devices 
designed to measure the “set” to respond in a socially desirable 
manner, viz, the Edwards and the Marlowe-Crowne scales; and 
(3) the factor defined by these latter measures may be tentatively 
identified with the second-order anxiety dimension found in pre- 
vious studies, 
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Tun Survey of Interpersonal Values (SIV) is a forced-choice 
inventory purported to measure basic motivational patterns or val- 
ues. Gordon (1960, p. 3) states that SIV “is designed to measure 
certain critical values involving the individual’s relationships to 
other people or their relationships to him. These values are im- 
portant in the individual's personal, social, marital and occupational 
adjustment.” 

Items for the SIV were selected through factor analysis, and 
triads of statements were assembled so that each statement keyed 
on one of six scales which were labelled Support, Conformity, 
Recognition, Independence, Benevolence, and Leadership. State- 
ments composing triads were equated for social desirability in 
order to reduce susceptibility to faking. Construction and defini- 
tion of the scales are described in detail in the SIV Manual (Gor- 
don, 1960). 

Gordon (1963) has presented extensive normative data and sum- 
maries of several exploratory studies relating SIV scores to occu- 
pational effectiveness, levels of professional training, scholastic 
achievement, psychiatric status, and delinquent behavior as well as 
to cross-cultural and sex differences. While these results appear 
logical and informative, neither theoretical formulations nor ex- 


1This research was supported by the Bureau of Medicine and Surgery, 


Department of the Navy, under Research Task MR005.12-2004, Subtask 1. 
Opinions or assertions contained herein are the private ones of the authors 
and are not to be construed as official or as necessarily reflecting the views 
of the Department of the Navy or of the Naval service at large. 

Appreciation is expressed to Mr. Frank A. Thompson and to Mr. Dale 
Trower for statistical computations used. 


121 


122 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


trapolations to relationships with other important aspects of life 
status have yet been offered. For example, no relationships between 
the SIV scales and such important life history and status variables 
as age, intelligence, educational level, socioeconomic or power 
status, social interests and activities, religious orientation, familial 
and cultural background have been reported. 

The present study is concerned with patterns of interpersonal 
values, as measured by the SIV, exhibited among military and 
civilian volunteers for the U. S. Antarctic Research Program (Op- 
eration Deep Freeze), and, within the military segment of this 
population, relationships of the SIV measures to a number of per- 
sonal history and status variables. 


Method 
Subjects 


Military and civilian applicants seen in psychiatric screening for 
Operation Deep Freeze over a two-year period were utilized for the 
study. Of the Navy population 7 percent were officers and 93 per- 
cent were enlisted men. Navy applicants averaged 26 years of age 
and ranged from one to 23 years in naval experience; 65 percent 
were high school graduates. Civilian volunteers averaged 29 years 
of age and ranged from one to 24 years in occupational experi- 
ence; 90 percent were high school graduates, and 65 percent were 
college graduates. Life history characteristics of Antarctic’ volun- 
teers have been described in detail in previous publications (Gun- 
derson, 1964; Nelson and Gunderson, 1963). 


Procedure 


Before deployment to the Antarctic each subject filled out a 
biographical inventory, the Personal History Booklet, and the SIV 
inventory. Subjects were exposed to essentially the same psycho- 
logical assessment procedures both years of the study. A number 
of items considered to be of most importance in summarizing life 
history events were selected from the Personal History Booklet 
and related to the SIV measures. 

SIV means and standard deviations were computed for the three 
Antarctic volunteer groups, Navy enlisted men, officers, and civil- 
ian scientists and technicians. Mean scores for these groups were 


Ps 
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compared with each other and with data published by Gordon for 
a number of other groups. Intercorrelations among the SIV scales 
were computed for the enlisted Navy sample and compared with 
those published by Gordon. Finally, for the total Navy population, 
13 items of life history and life status were related to the SIV scales 
by means of Pearson correlations and x? statistics, as appropriate. 


Results 


Table 1 gives means and standard deviations for five normative 
and professional groups drawn from the SIV Manual Supplement 
(Gordon, 1963), and for the three Antarctic volunteer groups. The 
differences in mean scores shown in Table 1 suggest relationships 
between social status, as reflected by educational and occupational 
attainment, and scores on certain of the SIV scales, Conformity 
and Benevolence appear to be negatively related to educational- 
occupational status, while Independence and Leadership appear to 
be positively related to the same variable. Similar relationships ap- 
pear to hold for the Antarctic volunteer groups on the Conformity 
and Independence scales. It does not seem surprising that the offi- 
cers emerged as highest of the Antarctic groups on the Leadership 
scale; it does seem surprising that Antarctic scientists scored high- 
est of any’group on the Benevolence Scale and lowest of any group 
on Recognition. 

The results in Table 1 suggest that Antarctic volunteers taken 
together have certain distinctive characteristics. Means for Con- 
formity and Benevolence are uniformly higher for Antarctic vol- 
unteer groups than for the normative and professional groups 
while means for Independence and Recognition are uniformly 
lower for the Antarctic groups. The magnitude of differences be- 
tween weighted means for the Antarctic groups versus those for 
the other groups reinforces the impression of a different pattern 
of values than is shown by other male groups.? Emphasis on social 


2 Precise tests for the significance of differences between the means of the 
total Antarctic population and the other groups combined were not possible 
since the exact variance for the combined groups was not known. However, 
if the largest sample variances are used as estimates of the population vari- 
ances, differences between the weighted means are highly significant for all 
six scales. Only the differences for Conformity, Recognition, Independence, 
no Benevolence appear large enough to be of any practical significance, 

owever. 
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conformity (Conformity) and helpfulness to others (Benevolence) 
as opposed to individual prominence (Recognition) and autonomy 
(Independence) would appear favorable for adaptation in small, 
long-term, confined groups such as those at Antarctic scientific 
stations, 

The data of Table 1 suggested a relationship between educational- 
occupational status and SIV values. It seemed plausible that age, 
education, rank (power status), or intelligence might account in 
part for the observed differences among group means. Pearson cor- 
relation coefficients were computed between these presumably con- 
tinuous variables and the SIV scales for the Navy volunteer popu- 
lation. The correlations of age with Conformity (r = .16) and 
Independence (r = —.15) were significant (p < .05) but in the 
opposite direction to that expected if age were to partly account 
for the differences in means found in Table 1. Education did not 
correlate significantly with any of the SIV scales for the Navy 
population. Rank correlated positively with Conformity (r = .10, 
p < .05) and Leadership (r = .16, p < .05). Intelligence (Navy 
General Classification Test) scores correlated significantly and in 
the expected direction with Conformity (r = —.24, p < .001) and 
Independence (r = .10, p < .05). It is noted that Gordon (1960) 
reported a significantly negative correlation between intelligence 
(The College Qualifications Test) and Conformity. It seems likely 
that the trends noted in Table 1 for the Conformity and Inde- 
pendence scales are not attributable to age or education; however, 
intelligence may account in part for the observed differences on 
those scales. 

The pattern of intercorrelations among the SIV scales shown in 
Table 2 generally conforms to that reported by Gordon for a col- 
lege student sample (1960, p. 5). Relationships among the scales, 
therefore, appear to be stable, being similar for these different 
populations, For both Navy and college populations the Support 
Seale correlates most highly with other scales, being substantially 
correlated with Recognition and Leadership. The preponderance 
of negative correlations is due to the forced-choice method pad 
resulting interdependence among scales, 

Table 3 presents an analysis of relationships between 13 life 
Status variables and the SIV measures for the total Navy sample, 
both officers and enlisted men (maximum N = 497). SIV score 
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TABLE 2 
Intercorrelations among SIV Scales 
Con- Recog- Inde- Benevo- Leader- 
formity nition pendence lence ship 
Support —268 46 -14 —20 -51 
Conformity —33 —43 28 —32 
Recognition —29 —34 —10 
Independence —42 00 
Benevolence ? —28 


Leadership 

* Decimal points are omitted. 
distributions were dichotomized as closely to the median score as 
possible providing “high” and “low” groups on each test variable. 
Life status variables were grouped into categories and related to 
the dichotomized SIV measures by the X technique. Certain differ- 
ences can be noted for the x? results compared with those for the 
Pearson correlations described earlier for age, education, and rank 
versus the SIV scales, suggesting non-linearity of these relationships. 

The Support scale was found to be significantly related to Rank. 
Officers scored lowest on this variable (35 percent above the me- 
dian score) while unrated enlisted men scored highest (66 percent 
above the median). Age and years of naval service also were sig- 
nificantly related to Support, although the forms of these relation- 
ships were curvilinear. Men in the 30-32 age category and in the 
9-11 years experience’ category, that is, persons falling near the 
middle of those distributions, tended to have lowest scores. 

Conformity was significantly related to rank; the relationship 
was distinctly curvilinear with the middle of the rank distribution 
scoring high on Conformity and the extremes low. 

Recognition was significantly related to Marital Status; the Sep- 
arated-Divorced category accounted for most of the variance as 80 
percent of this group scored above the median. 

Independence scores were not significantly related to any of the 
life status variables at the .05 level of confidence, 

Benevolence was related to rank, Deep Freeze experience, and 
frequency of worship. Benevolence scores tended to decrease with 
rank except for the first class petty officer group which scored 
relatively high. Men who had previously spent a year in the Ant- 
arctic tended to score low on the Benevolence scale (only 36 per- 
cent scored above the median), A distinctly linear relationship was 
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TABLE 3 


Relationships Between Life Status Variables and Gordon 
Survey of Interpersonal Values Scales* 


Variable df S Cc R I B L 
Rank 5 21.6 1195 3.2 10.5 14.65 14.8> 
Age T 159 79 92 107 68 126 
Years Service 6 15.76 7.9 7.5 9.0 118 6.4 
D.F. Experience 2 0.4 0.7 3.0 2.8 6.1> 1.7 
Education 5 7.3 3.5 5.4 TAEA 22. 
Religion 6 9.1 96 58 124 56 6.4 
Worship 3 3.8 6.2 5.5 40 8l 2.0 
Number of Siblings 2 2.0 4.0 1.5 1.0 0.0 3.0 
Family Mobility 6 2.6 BB ip I 18 2.2 4.4 
Rural-Urban Residence 4 4.2 8.1 6.6 3.2 1.0 7.3 
Marital Status 3 6.9 5.1 7.8 20 43 8.5% 
Parents’ Marital Status 4 24 3.5 1.0 2.9 3.5 11 
Region of Residence 8 8.9 3.3 4.4 8.1 4.3 7.8 


a x? statistics were computed from SIV scores dichotomized as near the median as possible 
versus life status variables grouped into varying numbers of categories; degrees of freedom (df) 
equal number of categories minus one. 

b x? significant beyond .05 level of probability. 


present between frequency of worship and Benevolence; those 
who worshipped regularly scored high and those who never wor- 
shipped scored low. 

The Leadership scale was significantly related to rank, education, 
and marital status. The relationship with rank was linear but dis- 
continuous in that officers were much higher than enlisted men on 
this variable (76 percent above the median), Education was highly 
related to Leadership scores; individuals with college experience 
scored above the median much more frequently (77 percent) than 
other categories. The “eleventh grade completed” category scored 
lowest on Leadership. Separated or divorced individuals tended to 
score high on the Leadership measure (64 percent above the me- 
dian). 

The number of siblings (none, one, more than one), family 
mobility, rural-urban residence, parents’ marital status, and region 
of childhood residence variables were not significantly related to 
any of the test variables. 


Discussion 
The findings indicated that values measured by the SIV inven- 


tory tend to be sensitive to differences in current social status and 
generally insensitive to differences in familial and cultural back- 


re 
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ground (parents’ marital status, number of siblings, region of child- 
hood residence, rural-urban residence, family mobility, and reli- 
gion.) 

The specific positive findings from the study appear very rea- 
sonable in retrospect: that low rank and inexperience are associ- 
ated with needs for support and encouragement is not surprising; 
that a college education should be correlated with willingness to 
exercise leadership seems predictable; that religious devotion 
should be associated with benevolent attitudes toward others seems 
congruent with expectations; and that there may be a higher prob- 
ability of marital friction and break-up where husbands are highly 
concerned about receiving attention and admiration and at the 
same time with being dominant and completely in charge does not 
appear illogical. Such relationships have not been previously docu- 
mented for the SIV; the findings provide some evidence for the con- 
current validity of certain of the SIV scales. 

Further support for some of the specific relationships observed 
in the present study is available from other sources. Gordon (1960) 
reported a correlation of .52 between the Religious Scale of the 
Allport-Vernon-Lindzey Study of Values and the SIV Benevo- 
lence Scale, indicating a close relationship between this measure of 
religious interest and benevolent attitudes. In a previous study, the 
authors (Gunderson and Nelson, 1964) noted that separated or 
divorced individuals scored significantly higher than other individ- 
uals on the Expressed Control Scale of the FIRO-B Inventory. 
Gordon (1963) reported a correlation of .60 between the SIV 
Leadership and the FIRO-B Expressed Control Scales for a sample 
of Antarctic scientists. These relationships tend to reinforce the 
plausibility of the common impression that particular syndromes 
of interpersonal needs or values in males may be inimical to suc- 
cessful marriage. The notion that regards sharp differences in in- 
terpersonal values as sources of marital incompatibility is not novel, 
but the specific clues suggested above may be worthy of further 
study. 

The fact that men who had previously spent a year in the Ant- 
arctic tended to score lower (64 percent of the 44 subjects in this 
category scored below the median) on the Benevolence scale than 
men without such experience is not readily explained from avail- 
able data, These men typically had experienced confinement and 
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enforced socialization for several months during the Antarctic 
winter and were again volunteering to spend a year living at close 
quarters. The possibility that benevolent attitudes undergo change 
in such an environment should be tested directly by obtaining 
measures on the same or comparable groups before and after Ant- 
arctic duty. It may be, however, that those actually selected for 
wintering-over assignments from among the many who apply 
score lower on the Benevolence scale or that a maturational process 
takes place such as that reported by Gordon (1963) for medical 
school students. 

The results have demonstrated that the Survey of Interpersonal 
Values scales are significantly related to certain important aspects 
of current life status. Further research with the SIV should take 
these relationships into account. 


Summary 


Relationships of 13 life history and status variables to the six 
scales of the Survey of Interpersonal Values (SIV) were examined 
in a population of Navy volunteers for duty in Antarctica. The 
Support scale was significantly related to age, experience, and 
rank while Benevolence was related to rank, previous Antarctic 
experience, and participation in worship. These and other results 
suggested that the SIV value scales were sensitive to differences in 
current life status but generally insensitive to differences in familial 
and cultural background. Since the SIV was designed to measure 
present value orientations, the study provided evidence for concur- 
rent validities of some of the scales. 
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CONTRASTED GROUPS VERSUS REPEATED 
MEASUREMENT DESIGNS IN THE EVALUATION 
OF SOCIAL DESIRABILITY SCALES! 

£ . 
GEORGE J. SKRZYPEK ann JERRY S. WIGGINS 
University of Illinois 


Socar desirability scales are psychometric indices which pur- 
port to tell us the extent to which individuals’ answers to per- 
sonality tests are influenced by a tendency for the respondents to 
place themselves in a favorable light. Wiggins (1959) compared the 
relative success of several such scales in differentiating between 
subjects instructed to “fake good” and other subjects not so in- 
structed on the MMPI. He found that some scales, such as his own 
and that of Cofer, Chance and Judson (1949) were able to separate 
such instructional groups with a modest amount of success while 
others, notably that of Edwards (1957) and the K scale (Meehl and 
Hathaway, 1946) were considerably less successful. Walker (1962) 
performed a partial replication of Wiggins’ study and obtained re- 
sults which reflected more favorably on Edwards’ scale (SD) and 
the K scale as social desirability measures. 

Since Walker (1962) employed different instructions and a test- 
retest design involving a single group of subjects it is not clear 
whether the discrepancies between his results and those of Wiggins 
should be attributed to the use of “improved” instructions as 
Walker (1962, 1963) maintains or to the use of a test-retest design 
as Wiggins (1963) asserts. Wiggins’ preference for a contrasted 
groups design was stated as follows: 

1 This study is based on a Masters thesis submitted to the University of 
Illinois by the first author and conducted under the supervision of the sec- 


ond author. The study was supported by Public Health Service 
Grant No. MH 07042-01 from the National Institute of Mental Health. 
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In addition to providing a more realistic basis for generalization, 
the contrasted groups design avoids the necessity of employing a 
test-retest control group for the “improved adjustment” effect that 
frequently occurs with repeated administrations of the same in- 
ventory (Windle, 1954, 1955; Wiggins, 1963, p. 110). 

Although both points of view in this controversy have been well 
explicated (Wiggins, 1959, 1962, 1963; Walker, 1962, 1963) the is- 
sues involved can be resolved only by additional experimentation. 
In the present study, improvement phenomena, contrasted groups 
versus test-retest differences, and the effects of different instruc- 
tions were investigated within the context of a single experimental 
design. 


Method 


One hundred and fifty male undergraduate students enrolled in 
an introductory course in psychology were administered the full 
scale MMPI on two separate occasions separated by a one week 
interval. The subjects were divided into three groups of 50 each 
according to the experimental design illustrated in Table 1, Self- 
report instructions were those printed on the standard MMPI test 
booklet. Fake instructions were designed to approximate as closely 
as possible those employed by Walker (1962) 2 


TABLE 1 
Experimental Design 
—————— Imum 


N First Administration Interval Second Administration 


50 Ia Self-Report one week Ib Fake 
50 Ia Fake one week IIb Self-Report 
50 Ila Self-Report one week IIIb Self-Report 


$ MMPI protocols were scored for the standard clinical and valid- 
ity scales as well as the six social desirability scales employed in the 
original study (Wiggins, 1959). These social desirability scales were: 
Edwards’ SD (Edwards, 1957), Hanley’s Tt (Hanley, 1957), Wig- 
gins and Rumrill’s Sd-A and Sd-R (Wiggins and Rumrill, 1959), 
Cofer’s Cof (Cofer et al., 1949) and Wiggins’ Sd (Wiggins, 1959). 
A brief description of the content and construction of these scales 
is given in Wiggins (1959). Scale means and standard deviations 


2 These instructions are given in Skrzypek (1964). 
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were computed for each group as a basis for t test comparisons 
among the groups. In addition, the proportion of hits and misses in 
group identification at different scale cutting scores was computed 
and the discriminative efficiency of each cutting score assessed by 
the phi coefficient. 

The design presented in Table 1 allows for several comparisons 
whose outcomes may be anticipated on the basis of previous experi- 
mental findings and the interpretations that have been placed on 
them. When the scale scores of Group Ia are compared with those 
of Group IIa, the contrasted groups design of Wiggins (1959) is 
replicated. The comparison of Group Ia with Group Ib is the test- 
retest design employed by Walker (1962). A comparison of Group 
Ib with Group IIa involves a direct test of differences between test- 
retest and independent experimental (faking) groups. Such differ- 
ences would be anticipated on the basis of the discrepant findings 
reported for Walker’s (1962) and Wiggins (1959) experimental 
groups. Since both experimental groups had identical instructions, 
differences between them would be attributed to previous experi- 
ence in self-report. 

An additional order effect may be tested by comparing Group 
Ia with Group IIb. The results of Voas (1958) suggest that prior 
faking experience may tend to result in more “honest” subsequent 
self report.’ In the earlier studies of Cofer et al. (1949) and Rosen 
(1956), no such order effects were obtained. 

A comparison between Group IIIa and Group IIIb allows for an 
evaluation of a possible “improvement” effect attributable to simple 
retesting. According to the manner in which Wiggins (1963) in- 
terprets Windle’s (1955) findings as applying to Walker’s (1962) 
study, such an order effect would be evident in Edwards’ SD scale. 


Results and Discussion 


Independent versus Retested Faking Groups 


Table 2 contains the results of the replication of Wiggins (1959) 
study (Group Ia versus Group IIa), the replication of Walker’s 


3 Although Voas (1958) required Ss to give self-report and fake answers at 

same point in time, his suggestion that bringing the tendency to fake to a 
conscious level may inhibit faking in self-report would seem applicable to 
the present design as well. 
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(1962) study (Group Ia versus Group Ib) and a direct comparison 
of the experimental groups from both designs (Group Ib versus 
Group Ila). In contrast to Wiggins’ original results, highly reliable 
differences between group means (Group Ia versus Group IIa) 
were obtained for all social desirability scales (including SD, K and 
F) in the present replication. Although the possibility of intrinsic 
differences in the subject samples employed in the two studies 
cannot be ruled out, Walker’s (1962) contention that his version of 
faking instructions inspires greater faking behavior, receives sup- 
port from the present findings. The replication of Walker’s study 
(Group Ia versus Group Ib) provides results similar to those origi- 
nally found, although the extent of mean differences in the SD scale 
is slightly less than that found by Walker (1962). 


TABLE 2 
Mean Scale Differences under Contrasted Groups and Test-Retest Designs 


Contrasted Groups Comparison 
Group Ia L F K SD Tt SdA Sd-R Cof Sd 


Self-Report X 2.60 5.86 13.78 29.66 11.48 24.34 24.84 11.92 13.14 
(N=50) s 1.96 404 434 6.03 2.93 7.68 330 3.07 3.77 
Group IIa = 9.30 3.96 20.02 36.30 18.68 33.88 29.18 23.90 27.36 
Fake 3.08 3.53 4.00 3.52 3.53 453 3.88 4.78 5.69 
(N = 50) t 12.85 248 7.40 6.66 11.00 7.49 5.97 14.77 14.58 

P 001 .02 001 .001 .001 .001 .001 .001 .001 


Test-Retest Comparison 
Group Ia L F K SD Tt SdA Sd-R Cof Sd 
Self-Report X 2.60 5.86 13,78 29.66 11.48 24.34 24.84 11.92 13.14 
(N=50) ~ 1.96 404 434 6.03 2.93 7.68 3.30 3.07 3.77 
GroupIb X 10.58 3.94 21.46 36.88 20.14 37.74 31.00 24.76 29.28 
Fake o 284 3.60 3.09 2.54 2.28 324 2.43 4.51 4.93 
(N= 50) t 15.43 3.12 12.26 8.40 18.36 9.83 11.59 18.58 19.97 
Pp 001 .01 001 .001 .001 .001 .001 001 .001 


Contrasted vs Test-Retest Experimental Groups 


Group Ia 
vB t 213 .03 1.99 94 243 1.08 2.78 92 1.78 
Group Ib p .05 .05 .05 01 


When scale scores from the two experimental groups (Group 
Ib and Group Ila) are compared, higher faking scores are found 
for all social desirability scales in the experimental group which 
had preceding self-report experience. These differences are reliable 
for only four scales (although K is one of them). The difference in 
SD scale scores for the two experimental groups is not a statistically 
reliable one. Nevertheless, it appears that faking groups with prior 
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self-report experience are not comparable to faking groups which 
have not had such experience. 


The Effect of Prior Experience 


The hypothesis that prior faking experience may lead to subse- 
quently more “honest” self-report was tested by a comparison of 
mean scale scores between Group Ia (whiċh had no prior experi- 
ence) and Group IIb (which had prior faking experience). With 
one minor exception, prior faking experience resulted in increases 
in social desirability scale scores rather than decreases as predicted. 
None of these differences was statistically reliable however. When 
scores on the nine MMPI clinical scales were compared between 
these two groups, decreases were found to have occurred in all 
scales. The only clinical scale difference that attained significance 
was that of Hs (p < .05). If anything, prior faking experience 
leads to decreased “honesty” in subsequent self-report. 

A comparison of Group IIIa and Group IIIb allows for an evalu- 
ation of a possible “improvement effect” which has been noted by 
Windle (1955) as occurring in test-retest self-report designs. In the 
present study, all of the social desirability scales increased on sec- 
ond administration in the absence of specific faking instructions. 
Edwards’ SD scale was the only scale which increased reliably 
(t = 2.25; p < .05) adding a dramatic note to Wiggins’ (1963) pre- 
diction of this finding. All of the clinical scales decreased (“im- 
proved”) on retesting. Only Hs and Hy did so significantly (p < 
-05). Studies such as Walker’s (1962) which evaluate the efficiency 
of Edwards’ SD by means of a test-retest faking design may be 
capitalizing on an improvement phenomenon which operates in the 
direction of the experimental hypothesis. 


Discriminative Efficiency 

Although the emphasis thus far has been on mean scale differ- 
ences between experimental and control groups, it should be clear 
that the ultimate basis for evaluating the usefulness of any screen- 
ing device is the extent to which such a device contributes to the 
accuracy of personnel decisions (Cronbach and Gleser, 1957). 
This, in turn, may be seen to be a function of the amount of over- 
lap which exists between experimental and control group score dis- 
tributions and the proportion of correct and incorrect decisions 
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made at a given cutting score (Meehl and Rosen, 1955). In Table 3 
are presented the proportion of correct and incorrect decisions 
made by each of the social desirability scales at the arbitrarily 
adopted cutting score at which both false positive and false nega- 
tive decisions are minimized. The relative success of these scales in 
correctly identifying fakers and non-fakers has now been suffci- 
ently replicated (Wiggins, 1959; Walker, 1962; Boe and Kogan, 
1964) to assume the status of a minor psychometric law. The pres- 
ent findings with respect to independent experimental and control 
groups are highly similar to those of the original study (Wiggins, 
1959) with the exception that all scales are more efficient, due per- 
haps to the use of Walker’s instructions. 

Although not presented, the findings with respect to discrimi- 
nation between the same subjects taking the test under self-report 
and faking instructions are highly similar to those of Table 3. 


TABLE 3 
Discriminative Efficiency of Social Desirability 
Scales in Contrasted Groups Design 
a ———— 
Group Ta (Self-Report) Group Ila (Fake) 
Proportion sie igs Proportion Proportion Phi 


called called called Coeff. 
Self-Report Fake Self-Report Fake 
ron Cut 

22+ -98 -02 -10 -90 -883 

Cof 18+ +96 04 10 -90 -862 

L T+ -96 04 16 84 -806 

Tt 16+ -90 -10 +16 .84 +741 

Sd-A 32+ -78 +22 12 88 -663 

Sd-R 30+ -98 02 +42 .58 611 

K 18+ -80 +20 +20 80 600 

SD 385+ -14 .26 -16 .84 583 
a ee EN 8k B88 


Hanley’s Tt did relatively better under test-retest conditions and 
the overall degree of discrimination (as reflected in the phi coeffi- 
cient) was greater for all scales, due perhaps to an improvement 
phenomenon. In both designs, Sd and Cof were among the three 
most efficient scales, while K and SD were the least efficient. 


Summary 


Wiggins ( 1959) , using an independent groups design in evaluating 
several social desirability scales, found certain scales to perform 
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considerably more effectively than others. Using different instruc- 
tions and a test-retest design, Walker (1962) obtained somewhat 
different results. The present study investigated the hypothesis 
that test-retest faking designs are subject to a spurious “improve- 
ment” effect attributable to simple retesting. It was found that the 
test-retest faking design results in higher faking scores than the 
design employing independent groups and this increment may, in 
part, be attributed to an “improvement” phenomenon. 
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INHIBITORY EFFECTS OF A PRETEST ON 
OPINION CHANGE* 


ROBERT E. LANA 
University of Rome, Italy 


In recently reported studies (Lana and Rosnow, 1963; Lana, 
1964b) it has been shown that the use of a pretest and its manner of 
presentation can affect the direction of opinion change when two 
opposed arguments on the same topic are utilized. Primacy refers to 
the success in changing opinion of an initial argument of two op- 
posed communications. Recency refers to a similar success of an 
argument presented second. When the pretest is hidden from the 
subject (Lana and Rosnow, 1963) or missing altogether (Lana, 
1964b), in the pretest-treatment-posttest opinion change research 
design (Solomon, 1949), change occurs usually in the direction of 
the first presented of two opposed arguments (primacy). Recency, 
however, does not necessarily occur when the pretest is exposed. 

The principal concern of this study is to examine the effect of the 
pretest on opinion change regardless of direction (primacy or re- 
cency). The initial focus of the studies mentioned above (Lana and 
Rosnow, 1963; Lana, 1964b) was on the effect of the present on the 
resulting directional effect (primacy-recency), but a more impor- 
tant consideration may be whether or not the pretest generally in- 


1 This research was supported by the National Institute of Mental Health, 
United States Public Health Service Grant No. MH 06926-02. 

The Vivisection Questionnaire, the pro and con communications, and the 
familiarization talk utilized in this study have been deposited with the 
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Auxiliary Publications Project, Photoduplication Service, Library of Congress; 
Washington 25, D. C., remitting in advance $1.25 for microfilm or $1.25 for 
photocopies. Make checks payable to: Chief; Photoduplication Service, Li- 


brary of Congress. 
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hibits or facilitates opinion change when two opposed arguments on 
the same topic are presented to the subject. The concept here is that 
the subject may “commit” himself to a particular position with re- 
spect to the topic at hand. Thus responding to a pretest should re- 
sult in less opinion change, either in favor of or against a specific 
issue, than where an initial pretest is not used. If this is true, then 


effects with IQ and sex in a similar training situation, Both utilized 
a single, unidirectional treatment. 

Hovland and Mandell (1957) have shown that commitment to 
an attitudinal position inhibits change when commitment is elicited 
after an initial influence attempt but not when Tesponse to a pre- 
communication questionnaire constitutes the commitment, Their 
Suggestion, however, is that there are forms of attitudinal commit- 
ment, usually made under public conditions, which inhibit opinion 
or attitude change as a result of materials presented later, 
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manipulating the nature of the pretest. The major question that 
remains concerns the nature of the relationship. At any rate, there 
seems to be ample information to establish the suspicion that initial 
responses of a subject in the opinion and attitude situation may 
very well depress later changes. 

In a study done in 1959, Lana found no depressive effect of the 
pretest (interaction between pretest and treatment) when a single, 
unidirectional communication was utilized. However, when two op- 
posed communications on the same topic were presented to the same 
subject (Lana and Rosnow, 1963, 1964b) a pretest inhibition or de- 
pressive effect was present, Recalculation of the data of these studies 
was necessary, and the depressive effects were not reported at the 
time of publication. 

In order to facilitate comparisons among the various experiments 
already reported in the literature and the current one, the procedure, 
results, and conclusions of each of the three previously published 
experiments are listed below. All primacy-recency effects are 
summed by taking the algebraic differences between pretest and 
posttest and subtracting these differences for each pro-con and con- 
pro group. These results have already been reported in the first three 
published studies cited below and, for the fourth study, in Table 2 
of this report. All opinion change effects regardless of direction 
(primacy or recency), are absolute differences, taken between pre 
and posttest for all groups and averaged regardless of algebraic 
sign. 


Experiment I (Lana 1959) Exposed Pretest, Unidirectional Per- 
suasion. 


One-hundred fifty-six college students were randomly assigned to 
four treatment conditions. Two of these groups received the pretest 
attitude questionnaire. One of the two groups listened to a taped 
pro-vivisection communication 12 days after taking the pretest. 
After treatment, this group (Group I) was immediately posttested 
with the same questionnaire. The other group (Group IV) was sim- 
ply posttested 12 days later. Group II heard the pro-vivisection 
communication and was posttested immediately afterward without 
having been pretested. Group III answered the questionnaire once. 
Opinion change was measured as the algebraic difference from pre- 
test to posttest. 
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The only significant F was that for the treatment effect, while the 
F ratios for the effects of the pretest and the pretest-by-treatment 
interaction were both less than unity. It can be concluded that the 
pretest and treatment did not interact, and that the pretest did not 
sensitize or desensitize the subjects to the communication. The 
main effect of pretesting was not significant. 


Experiment II (Lana and Rosnow 1963) Exposed Versus Hidden 
Pretest, Bidirectional Persuasion 


One-hundred twenty-eight students from beginning level psy- 
chology classes at the American University were used as subjects in 
this study. They were randomly divided into eight groups of 16 
each. All S's initially received either the opinion questionnaire on 
nuclear weapons or one on public censorship. Four of the groups 
were presented the pretest during a regularly scheduled class as a 
separate part of the hour's activity. These were labeled the “ex- 
posed pretest” group. The remaining four groups responded to the 
questionnaire presented as part of their regularly scheduled hour 
examination in general psychology. The opinion questions appeared 
as a 5-question unit in the exam and were “buried” among other 
similar question units pertaining to the usual subject matter of an 
introductory psychology class. These groups were labeled the “hid- 
den pretest” group. All groups, immediately after listening to the 
pro and con arguments, received the appropriate posttest, which 
was identical with the pretest. 

Primacy-recency effects are given in the 1963 report on this ex- 
periment. Recalculation of the data indicated that the average 
opinion change per group (mean absolute differences from pretest 
to postest) of the four groups where the pretest was hidden was 
1.88 scale points as compared with an average absolute change of 
Al scale points per group of those four groups where the pretest was 


exposed. This difference was significant beyond the .05 level (t-test 
for independent means). 


Eevee III (Lana 1964b) Exposed versus No Pretest, Bidirec- 
tional Persuasion 


One-hundred thirty-six subjects from the introductory psychol- 
ogy classes at Alfred University were randomly assigned to 12 ex- 
perimental groups. Two of the groups were “zero” control groups 


quu — 
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which received the pretest and then the posttest either 12 or 24 days 
later. Neither of these groups indicated any change in opinion what- 
soever and thus were dropped from any further analysis. Of the re- 
maining 10 groups, eight were pretested with an opinion question- 
naire on Nikita Khrushchev. Pretests for all groups were then 
analyzed for differences with a one-way analysis of variance to de- 
termine whether or not the groups could reasonably be assumed to 
be homogeneous with respect to initial opinion concerning Khrush- 
chey. The F ratio was less than 1. It was concluded that the 
groups were homogeneous on initial opinion. It was assumed that 
initial opinions of the two groups not pretested were homogeneous 
with the eight pretested groups since they were formed randomly 
from the same sample from which these latter groups were selected. 

Four of these eight groups, immediately after filling out the pre- 
test, were exposed to the successive presentation of two opposed 
(pro and con) prose statements regarding Khrushchev. Two of these 
groups received the pro argument first and the other two groups the 
con argument first. Two of these four groups were then immediately 
posttested. The remaining six groups, and the two groups that were 
not initially pretested, were dismissed for 12 days. At the termina- 
tion of this interval, all eight groups who had not as yet been given 
a posttest were run as follows: The two groups that had been ex- 
posed to the pro and con arguments were posttested with the identi- 
cal questionnaire used as the pretest. The two groups receiving no 
pretest were givensthe pro-con or con-pro order of arguments and 
immediately posttested. Two of the groups having received the pre- 
test only received the pro-con or con-pro order of arguments and 
were then posttested. The remaining two groups, who had received 
the pretest only, were exposed to the arguments as had the other 
groups. They were then posttested 12 days after the presentation of 
the arguments. Because this design is more complicated than those 
of the other reported studies, a summary of the experimental pro- 
cedure is given in Table 1. 

In the two groups receiving no pretest, the average opinion 
change (mean absolute difference from pretest to posttest) per 
group was .75 scale points as compared with an average per group 
change of .85 scale points for the groups receiving a pretest. These 
differences are significant beyond the .05 level (t-test for independ- 
ent means). 
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TABLE 1 
Experimental Design for Experiment III 


GROUPS 
As iB Cr SD) E F LOM BST I J K L 
Pretest Pretest Pretest Pretest Pretest 


Pro Con Pro Con Pro Con 12 days 12 days 12 24 
Con Pro Con Pro Con Pro Pro Con Pro Con days 


Posttestı 12 days Posttest; Posttestı 


Posttest; 
3-months delay 
Posttests 
Mii ic. 


Thus, in Experiment I, with a unidirectional influence attempt, 
no damping of the posttest by the pretest was found. In Experiment 
II, with a hidden pretest and bidirectional persuasion, there was no 
damping effect of pretest on posttest, but there was a damping effect 
when the pretest was exposed. In Experiment III with bidirectional 
persuasion and no pretest, there was no damping effect (estimated) ; 
with an exposed pretest, there was a damping effect. For procedural 
details and analysis of primacy-recency effects the original publica- 
tions should be examined. 


Experiment IV—Current Study 


The primary hypothesis is that a pretest will dampen the effect of 
bidirectional persuasion. A second hypothesis was also tested in this 
study. Previous research has shown that a subject familiar with the 
topic of a communication is more likely to change his opinion in the 
direction of the first presented of two opposed arguments (primacy 
effect) when that topic is concerned with some social issue, As 
stated above, it also seems that a primacy effect is prevalent in 
cases where the subject is not pretested. In combining these two 
conditions, high familiarity and no pretest, a pronounced primacy 
effect should occur, 

One-hundred subjects in the introductory psychology classes at 
Alfred University were tandomly assigned to one of eight experi- 


= 


= 
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vivisection.” Only about ten percent of all subjects knew to what 
vivisection referred. This group, and randomly chosen subjects 
from the unfamiliar students, were designated the “familiar group.” 
The remainder of the subjects were designated the “unfamiliar 
group.” Just prior to pretesting, and again just prior to posttesting, 
the familiar group was read a 213 word passage describing vivisec- 
tion. The information presented was factual and did not represent 
any attitudinal point of view. Thus the familiar group, some of the 
members of which already possessed some knowledge of the topic, 
was exposed to information about the nature of vivisection. The 
familiar and unfamiliar groups were randomly divided, half receiv- 
ing a questionnaire tapping opinions for or against the practice of 
vivisection. These groups were again randomly subdivided into 
groups receiving a pro-vivisection communication followed by a 
con-vivisection communication, or the con-communication and 
then the pro-communication. The scores on the pretest for the four 
groups were analyzed for differences by a one-way analysis of 
variance to determine whether the groups could reasonably be as- 
sumed to be homogeneous with respect to initial opinion on vivi- 
section. The F ratio was less than one. It is concluded that the 
groups were homogencous on initial opinion. It is assumed that the 
four remaining groups not pretested are homogeneous with the four 
pretested groups since they were formed randomly from the same 
sample as the pretested groups. In a single study such as this one, 
assumptions of this nature might seem tenuous. However, in other 
studies (Lana, 1959, 1964b) there seemed to be reasonable assur- 
ance that one could assume that unpretested groups had similar ini- 
tial opinions as pretested groups. Random allocation assured homo- 
geneity of opinion for the variously formed experimental groups 
over the long run. In any case, random allocation is one of the few 
techniques available at this time which is useful in creating the 
conditions necessary for making a test of the current hypothesis. 
One alternative to administering a pretest, and being reasonably 
certain of the opinion of the group on some topic, is to use groups 
which have been formed because of their stand on the very topic in 
question. Other difficulties, however, arise with this technique 
(Lana, 1964b). 

All groups were then posttested with the identical questionnaire 
used as the pretest. The design is presented in Table 2. 
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Materials 


A Likert-type, 19-item, 6-alternative questionnaire (Molnar, 
1955) was utilized as both the pretest and posttest. It has been used 
in two previous studies by Lana (1959, 1961). High score indicates 
Pro-vivisection opinion. The range of possible scores is 6 to 60. The 
pro and con communications on vivisection are about 500 words 
long and take about five minutes to read. They were developed by 
Molnar (1955) and have succeeded in changing the opinions of 
college students toward vivisection in other similar studies (Lana, 
1959, 1961; Molnar, 1955). 

Vivisection was chosen as the topic since it is generally not con- 
sidered particularly controversial (Lana 1962) by most college 
students. Also, from the results of the initial questions asked, many 
students have no idea as to what the term refers. This makes it easy 
to familiarize the appropriate groups with the topic by providing 
vivisection information to them, thus increasing the probability 
that the familiar and unfamiliar conditions have been established 
in the experimental groups. Zero control groups (no communica- 
tion) were not included in the design because in all similar instances 
(Lana, 1959, 1964b) no pretest-posttest changes were evident, 


Results 


The results of the experiment were analyzed by the use of the 
t-test for significance of average absolute differences between pre 
and posttest and the subtractive-difference technique, which is use- 
ful for detecting directional effects in opinion change. The groups 
not pretested were assigned mean pretest values based on the aver- 
age of the scores of the pretested subjects. The average absolute 


Significant (01) ‘Tecency effects were evident in the familiar 
groups, and no significant directional effects of any kind resulted 
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in the unfamiliar groups. There were no differences in significant 
primacy-recency effects between the pretested and unpretested 
groups. The results are summarized in Table 2. 


TABLE 2 
Experimental Design and Summary of Primacy—Recency Results 


2 Questions on Vivisection—S Days 


FAMILIAR UNFAMILIAR 
INFORMATION ON NO INFORMATION ON 
VIVISECTION VIVISECTION 
NO PRETEST PRETEST NO PRETEST PRETEST 
12 DAYS 
PRO CON PRO CON PRO CON PRO CON 
CON PRO CON PRO CON PRO CON PRO 
Pe DAE TNE ba ANEA 
INFORMATION ON NO INFORMATION ON 
VIVISECTION VIVISECTION 


———— 


————— 


Posttest 
(EST.) (EST.) (EST.) (EST.) lss 
PRE 37.43 37.43 37.00 35.71 37.43 37.43 39.92 37.36 
POST 36.18 39.29 36.45 38.07 40.00 40.45 39.33 37.07 
GAIN —1.25 +1.86 —.55 +2.36 +2.57 +3.02 —.59 —.29 
DIFF. OF 
DIFFER- 
ENCES —3.11 —2.91 —.45 —.30 
(recency) (recency) (recency) (recency) 
n 11 14 11 14 13 11 12 14 
t **3.99 **4.04 <1 <1 
*p <01 
Discussion 


The results indicate that the first hypothesis has been supported. 
Opinion change was significantly greater under conditions where no 
pretest was utilized than under conditions of pretesting. Unpre- 
tested, disguised pretest, and pretest groups (Lana, 1964b; Lana 
and Rosnow, 1963), and groups in preliminary experiments to the 
current study, were examined for differential amounts of opinion 
change. In all cases the no-pretest and disguised pretest groups 
changed to a greater degree than the pretested groups. This rep- 
resents observations on 520 subjects. There seems to be some sup- 
port for the contention that the pretest can act as a device by which 
the individual commits himself to maintain his opinion in the face 
of opposed (bidirectional) arguments presented later. Presumably, 
a single, unidirectional communication allows for greater change 
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regardless of initial pretest conditions because, if the individual is 
initially in favor of the position advocated by the communication 
his initial commitment is supported. If about half of all subjects are 
in favor of a given argument (assuming a normal distribution of 
pretest scores concerning the issue at hand, an assumption which 
holds true for this study) this half need not consider their initial 
commitment. On the other hand, when two opposed arguments are 
presented, there will always be one position confronting every sub- 
ject which is essentially opposed to his own initial commitment, 
Thus, his commitment via pretest may influence his reaction to ma- 
terial presented later in the experiment, and this will be true for all 
subjects. 

The pretest seems most effective in minimizing opinion change 
when the subject is later confronted with two opposed arguments on 
the topic in question rather than when he is later presented with a 
single, unidirectional argument. Conceivably one may speculate that 
the more complex and multi-sided the communicative materials, the 
greater the inhibitory effect of the pretest. Also it may be possible 
that the length, complexity and obviousness of the intent of the 
pretest might produce less opinion change after a communication 
than when a pretest is relatively simple, short and disguised. 


The topic of vivisection has characteristically been rated as one 
of low controversy by college students, It has been shown (Lana, 


E ow controversy tend to produce no 
significant prmacy-recency effects, This might have mitigated 


we — 
a eee 
teeing 
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against the appearance of primacy throughout all the groups of the 
study. It is also obvious that the familiarity of the subjects was 
more important in producing order effects than the fact of having 
been, pretested or not. In the current experiment, unlike previous 
studies on familiarity (Lana, 1961; Rosnow and Lana, 1965), the 
familiarization talk was presented twice, once before the pretest and 
again before the communications. This, and the questions used to 
measure existing familiarity in the subjects towards vivisection, 
probably provide a better assurance of familiarity for the subjects 
of this study than for those of previous studies. Even with these 
differences in methodology between the present study and the others, 
there is no convincing reason why recency should have dominated 
under conditions of familiarity. 


Summary and Conclusions 


One-hundred subjects were exposed to various experimental con- 
ditions where familiarity-unfamiliarity and pretesting-no pretesting 
were manipulated to examine their influence on magnitude of, and 
order effects in, opinion change. The following conclusions were 
made: 

1. The taking of a pretest in the form of an opinion questionnaire 
inhibits opinion change regardless of the direction of change 
(primacy-recency) that may be strongest when opposed argu- 
ments are utilized. 

2. Familiarity of the subjects with the topic of the influence attempt 
does not necessarily result in a significant primacy effect as pre- 
dicted. 

Theoretical interpretations of primacy-recency opinion change and, 

perhaps, measurement methodology, need to be revised and sensi- 

tized to establish consistent, successful predictions of these effects. 
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INDIVIDUAL PROFILES! 
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In clinical medicine the symptoms of a patient are matched to a 
disease syndrome to establish the initial diagnosis. The clinical 
psychologist for many years has tried to find test patterns which 
will have diagnostic value so that he, too, can match the patient's 
test profile (symptoms) to a diagnostic model. While the approaches 
utilize the same method, the practical aspects of the tasks differ. 

In clinical medicine, each disease usually has a single syndrome 
picture, occasionally with variants, so that a stable, useful model 
can be established and applied uniformly in practice. If an internist 
states that a patient showed the usual symptoms (syndrome) of 
obstructive jaundice, another would know what symptoms were im- 
plied. When a psychiatrist states that his patient showed the usual 
symptoms of schizophrenia, another psychiatrist would know only 
that the patient was psychiatrically ill and would be uncertain 
which symptoms might be present and which absent. 

As psychologists, we must assume that a number of different 
models will be required to describe the members of a single diag- 
nostic group such as those having “schizophrenia.” The notion of 
isomorphic correspondence between psychiatric classification and 
symptom pictures may be a convenient simplification, but it is un- 
realistic. 

Since Wechsler intelligence scales have been prominent in the 


1The author is indebted to Drs. Douglas E. Scates, Ralph M. Reitan, 
Clayton E. Ladd, and Frank M. du Mas for their critical reading of the manu- 
script, Dr. Scates was particularly helpful in clarifying the term, mean profile. 
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search for diagnostic patterns, it is appropriate to turn to Wechsler’s 
earliest proposals of patterns. Wechsler (1944) proposed a single 
pattern for each psychiatric diagnosis. During the subsequent 15 
years he has not acknowledged in his manuals the need to propose 
multiple patterns for any of the diagnostic groups (Wechsler, 1958). 

Those who try to test the validity of Wechsler’s proposed patterns 
fail to find any description of how they were derived. It is usually 
assumed, since there is only one pattern for each diagnosis, that the 
pattern is a profile of the mean subtest scores for a sample of pa- 
tients from that diagnostic group.? But the profiles of the mean sub- 
test scores of actual samples of patients are almost certain to show 
much less variance (scatter) than found in Wechsler’s patterns. 
Possibly the patterns proposed are those that he felt were represent- 
ative of Ss most typical of each diagnostic group. If this is so, 
Wechsler has presented modal patterns? rather than profiles of the 
means for actual groups. 

Modal profile means a hypothetical profile of test scores that is 
typical of a number of individuals in a diagnostic group. It is the 
average profile obtained from a tight cluster of individuals with 
congruent profiles. The term modal refers to the frequent recurrence 
of the profile for members of the diagnostic sample. It should not be 
supposed that the statistic, mode, enters its derivation. In a com- 
putational sense the modal pattern 7s a profile of mean test scores 
in a battery. However, the mean Scores are computed for a modal 
group (cluster) of similar profiles with the result that mean scores 
will not be intermediate values that seldom or never occur for 
actual subjects. 

It is held that only modal patterns are suitable for use as diag- 
nostic models. A modal pattern is a close approximation to a number 
of profiles actually obtained by Ss. The usual profile of the means, 


profiles of the individual Ss had been avera, 
b ged. 
3 Throughout the paper an arbitrary, but useful, distinction is made between 


significance. The pattern i i £ 
profile is matched. aa aaie 
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on the other hand, is a whole-group statistic and every individual 
in the group can have a profile quite dissimilar to the profile of the 
means. It is desirable that we concentrate on the development of 
modal patterns, which may typify individuals, since diagnosis deals 
with individuals, not with groups. Profiles of the means from diverse 
subjects have little value in diagnostic investigations. 

Wechsler probably failed to recognize clearly that his proposed 
patterns were intended to serve as modal patterns rather than pro- 
files of the means or he would have seen the need to present more 
than one pattern for each group. Or, he may have hoped that others 
would work in this badly understood area to develop multiple modal 
patterns for each diagnostic group from quantified data. But there 
has been almost no progress in the development of such multiple 
modal patterns; in fact, modal patterns have failed to receive con- 
sideration, while study after study presents analyses in terms of the 
profiles of the means. The absence of quantitative techniques for 
analysis of profiles is sufficient to account for the neglect of modal 
patterns by investigators. 

Except for the procedure suggested by Saunders and Schueman 
(1962), useful techniques of analyzing out modal patterns from 
masses of data have not been forthcoming. Considerable progress 
has been made in our thinking about the very fundamental question 
as to what is meant by similarity in profiles (Cattell, 1949; Cron- 
bach and Gleser, 1953; du Mas, 1949; Haggard, et al., 1959). 
Without such clarification in our thinking, we cannot proceed to 
find modal patterns which depend upon clusters of similar profiles 
for derivation. 

Should profiles be similar in shape, or level, or variance (scat- 
ter)? or any two of these? or all three? There is no simple answer 
because the importance of any of these three features is a function 
of the particular diagnostic circumstances. For example, it is un- 
realistic to expect all hysterics to have the same overall native in- 
tellectual endowment, and if a S’s Wechsler profile fits the variance 
and shape of the pattern but not the level, we could not conclude 
that he was not a hysteric. On the other hand, perhaps a type of 
left-hemisphere brain lesion will lower the level of functioning to a 
fairly constant level for all such cases despite original level; then 
level might prove to be a very important differentia for this modal 
pattern. Each clinical category studied requires a re-evaluation of 


154 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


how the similarity criterion should be established, but all three 
features of the profiles must receive some initial consideration, 

The usual subtest scatter scores (deviation of each subtest score 
from the individual’s average level of functioning) provide a profile 
that retains shape and variance but ignores level. Product-moment 
correlation of two profiles of subtest scores also ignores level but 
also disregards differences in average variance (amount of scatter) 
in the two profiles, Rank-difference correlation coefficients com- 
pletely ignore differences in variance (as well as level) in subtest 
Scores so that the profile of a normal person (low scatter) can cor- 
relate very highly with a badly deteriorated organic patient (large 
subtest scatter). For example, the profile of mean Wechsler subtest 
Scores of Rappaport’s normal control group correlated .90 (rank- 
difference method) with the profile for a group of deteriorated, 
diffuse brain-damaged patients (Matthews, Guertin, atid Reitan, 
1962). 

Some statistics have been Proposed that attempt to consider 
shape, level and variance in arbitrarily weighted degrees when com- 
paring two profiles. While shape is a prime criterion for pattern 
similarity for most diagnostic groups, it is unlikely that correla- 
tional techniques alone will identify diagnostic modal patterns, 


Illustration 


The general problem of finding clusters of similar profiles in a 
diagnostic sample of people is reducible to two phases. First, is the 
mathematical definition of similarity, Secondly, these indices of 


similarity are examined through matrix algebra to establish bounds 
on the clusters of similar Profiles, 


There is some similarity between 
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Product-moment correlation coefficients depict similarity of pro- 
files with respect to shape only. Thus, in Figure 1, profiles 1, 2, and 
3, will be perfectly correlated with one another as well as with pro- 
files 4, 5, and 6. Yet it is obvious that while the six profiles are 
similar in shape, they are not congruent because they are split into 
two markedly different levels, . 

The class of distance measures, such as Mahalanobis’ D-Square, 
(1936), reflect similarity with respect to level and variance as well 
as shape. These distance measures are indices of the difference in 
heights of two profiles at each.of the subtest score points. Distance 
measures come closer to representing what is meant by saying that 
two profiles are similar. Yet when a D-Square for two profiles is 
large it does not give any hint as to why the profiles are dissimilar. 
A large D-Square may arise from differences in shape, level, or 
variance, and, of course, any combination of these. 

One unfortunate characteristic of the D-Square index is that two 
profiles with opposite shapes will yield a small D-Square when they 
are at the same levels and variance is small. If, as experience sug- 
gests, shape is of prime importance in classifying people from their 
profiles, the sole dependence on D-Squares will obscure some modal 
patterns. ` 

Most psychologists are familiar with the factor analysis of cor- 
relations between people on a number of variables or test scores. 
Such an analysis is variously called transposed, inverted or Q 
factor analysis. Stated in terms of the pattern problem, the factor- 
ing of inter-profile correlations produces factors (clusters) of sim- 
ilarly-shaped profiles. Level and variance differences are, of course, 
ignored. yi ` i 

i Factor analyses of distance measures are rarely reported in the 
literature. But enough is known to expect them to produce clusters 
of profiles similar with respect to level and variance, and usually 
with respect to shape as well. (Nunnally, 1962; Overall, 1964) 

The procedure to be described herein, first factor analyzes inter- 
profile Correlation coefficients. Then, the similarly-shaped profiles 
are examined by D-Square comparisons to see if they split into two 
or more clusters on the basis of levels and/or variance differences. 

For example, in Figure 1, the perfectly correlated profiles 1, 2, 3, 
4, 5, and 6 should split into two clusters at different levels when the 
inter-profile D-Squares for the six profiles are factor analyzed. 
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The same is true for profiles 7, 8, 9, 10, 11, and 12. The procedure 
now to be described is eminently successful in doing this. 

A completely automatic computer package has been prepared by 
the author to do the successive computations described and illus- 
trated here. Output from the computer is lengthy, but the essential 
results will be presented below from the analysis of the dummy 
problem represented by the profiles in Figure 1. 

Two distinct shapes of profiles are seen in Figure 1. They are 
superimposed upon one another at the upper and at the lower level. 
The intercorrelation matrix, as seen in Table 1, contains only zeroes 
and ones: 

The reduced intercorrelation matrix is factor analyzed by the 
Principal Axis method and rotated to the Varimax criterion to give 
the factor matrix in Table2. , 

TABLE 1 
Reduced Intercorrelation Matriz with Communalities in the Diagonals 


1 2 3 4 5 6 7 
1 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0 
2 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0 
3 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0 
4 1,0000% 1.0000 1.0000 1.0000 1.0000 1.0000 0. 
5 1.0000 “13 1.0000 1.0000 1.0000 1.0000 0. 
6 1.0000 1. 1.0000 1.0000 1.0000 1.0000 0. 
7 FO 0. 0. 0. 0. 0. 1.0000 
8 0. oe 0. 0. 0. 0. 0. 1.0000 
9 0. gi 0: 0. 0. 0. 0. 1.0000 
10. 0s 0. 0. 0. 0. 0. 1.0000 
11 Sas 0. 0. 0. 0. 0. 1.0000 
127 70h 0. 0. 0. 0. 0. 1.0000 
13. eae 0. 0. 0. 0. 0. 0. 
14.5885 0. 0. 0. 0. 0. 0. 
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Teo 0. 0. 0. 0. 0. 0. 
Boat ci Oe 0. 0. 0. 0. 0. 

Seer iO; 0. 0. 0. 0. 0. 0. 

4 0. 0. 0. 0. 0. 0. (ides 
Due 305 0. 0. 0. 0. 0. 0. 
(Shas 0. 0. 0. 0. 0. 0. 

7 1.0000 1.0000 1.0000 1.0000 1.0000 0. 0. 

8 1.0000 1.0000 1.0000 1.0000 1.0000 0. 0. 

9 1.0000 1.0000 1.0000 1.0000 1.0000 0. 0. 

10 1.0000 1.0000 1.0000 1.0000 1.0000 0. 0. 

11° 1.0000. 1.0000 1.0000 1.0000 1.0000 0. 0. 

12 1.0000 1.0000 1.0000; 1.0000 1.0000 0. 0. 

134 0. 0. 0. Tano. 1.0000 —1.0000 
14 208 0. 0. 0. 0. —1.0000 1.0000 
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TABLE 2 
Rotated Factor Matrix from the Intercorrelated Profiles 
Rotated Factor Loadings 
Profiles I I Tr 
1 1.0000 0. 0. 
2 1.0000 0. 0. 
3 1.0000 0. 0. 
4 1.0000 0. 0. 
5 1.0000 0. 0. 
6 1.0000 0. 0. 
7 0. 1.0000 0. 
8 0. 1.0000 0. 
9 0. 1.0000 0. 
10 0. 1.0000 0. 
11 0. 1.0000 0. 
12 0. 1.0000 0. 
13 0. 0. —1.0000 
14 0. 0. 1.0000 


The next stage is to analyze separately each of the factor columns 
in the rotated matrix to identify which profiles have high loadings 
on each of the factors. Factor columns are dropped unless three 
profiles have sizeable loadings (.40). This stage controls all future 
analyses because it feeds the information about loaded profiles for- 
ward for D-Square computation, factor analysis and computation 
of modal patterns. Each shape factor is completely analyzed 
through to the identification of modal patterns before it comes back 
here for the processing of the next shape factor. 


TABLE 3 


Transformed D-Squares among Profiles That are Loaded on the 
First Shape-Family Factor 


” Profiles 


6 


-0000 285.0000 64.0000 33.0000 0. 

-0000 288.0000 93.0000 64.0000 33.0000 
-0000 288.0000 120.0000 93.0000 64.0000 
i 93.0000 120.0000 288.0000 288.0000 285.0000 
-0000 64.0000 93.0000 288.0000 288.0000 288.0000 
33.0000 64.0000 285.0000 288.0000 288.0000 


288 
288 
288 


The next stage is to compute D-Squares by the formula: 


L 
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Where X is the raw score on the subtest j; and 7 and m are the two 
profiles being compared; and L is the number of subtest scores. 

The D-Squares are reversed in direction by subtracting each 
from the largest obtained D-Square (289.00). After transforma- 
tion, the largest raw D-Square will have a value of zero. The largest 
transformed D-Square of each column serves as the estimate of 
communality and is inserted in the diagonals in Table 3. 

Next is the factoring and rotation of the transformed inter-profile 


` D-Squares. The rotated factor matrix is shown in Table 4. Here we 


see that the loadings on the D-Square factors indicate the presence 
of two patterns—one for profiles 4, 5, and 6; and the other for pro- 
files 1, 2, and 3. 


TABLE 4 


Rotated Factor Matrix Obtained from Analyzing Transformed 
Inter-Profile D-Squares for Profiles That are 
Loaded on the first Shape-Family Factor 


Factor 
Profile 
1 0.0474 17.0355 
2 1.8951 16.8841 
3 3.6370 16.6268 
4 16.6273 3.6378 
5 16.8842 1.8951 
6 17.0353 0.0470 


Keep in mind we are still analyzing only the first shape family 
of profiles, corresponding to the first column of the rotated factor 
matrix (Figure 2) derived from the intercorrelations of the raw pro- 
files. Now, the D-Square factor matrix is searched for profiles loaded 
heavily on the factors. Two profiles must be loaded heavily, or 
three moderately to establish a modal pattern. Pivot tests in the 
first column of the factor matrix are identified and then the next 
stage computes the modal pattern. 

This next stage computes the scores which describe the modal 
pattern. The values are the mean scores of the pivot profiles (num- 
bers 4, 5, 6) on the variables, weighted in relation to the size of the 
D-Square factor loadings of each of the pivot profiles. 

The four obtained score values for the first modal pattern are: 
21.02, 17.02, 21.02, and 17.02. These correspond closely to the true 


values of Figure 1, that are: 21.00, 17.00, 21.00, and 17.00. 


‘After the first modal pattern has been identified, we go back to 
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the stage that analyzed the output of the D-Square factoring. There 
the pivot profiles are picked up that identify the modal pattern in- 
herent in the second column of the D-Square factor matrix. These 
pivot profiles are used for computating the second modal pattern. 
The second modal pattern has the following score values: 5.98, 1.98, 
5.98, and 1.98. True values in Figure 1 differ from these again by 
only .02 raw score points. 

Only now after identifying the two modal patterns inherent in 
the first shape factor, do we return to analyze the second shape 
factor. D-Squares are computed for profiles loaded on the second 
shape factor and are then factored. The D-Square matrix is not re- 
ported here but the results of factor analyzing the profiles heavily 
loaded on the second shape-family factor are given in Table 5. 


TABLE 5 
Rotated Factor Matrix Obtained from Analyzing Transformed 
Inter-Profile D-Square for Profiles That are 
Loaded on the Second Shape-Family Factor 


Factor 
Profile 
a 0.0474 17.0355 
8 1.8951 16.8841 
9 3.6370 16.6268 
10 16.6273 3.6378 
11 16.8842 1.8951 
12 17.0353 0.0470 


The third modal pattern is as follows: 17.02, 17.02, 21.02, and 
21.02. The fourth modal pattern is: 1.98, 1.98, 5.98, and 5.98. Once 
again the results differ only .02 score points from the values in 
Figure 1. 

An almost exact solution was obtained for the profiles in Figure 
1. But what about clusters of profiles that are much closer in level 
than in Figure 1, will the procedure still identify them correctly? 

Figure 2 presents 12 profiles, similar to those in Figure 1 but the 
difference in levels between modal patterns is less. The four modal 


profiles were correctly identified and score values obtained for each 
profile were: 


Pedr 
rae 
Ssg 


ae 


12 


WILSON H. GUERTIN 
, F 


2 3 
SUBTESTS 
Figure 2. 


S 


161 
12 


11 


10 


162 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Figure 3 presents 14 profiles with four patterns distinguishable 
on the basis of variance, level and shape. These modal profiles were 
correctly identified and the subtest score values obtained for each 
score: 

1. 1.87 1.87 8.87 8.87 
2. 5.13 5.13 6.13 6.13 
3. 26.87 19.87 26.87 19.87 
4, 1 124113 -28.13 24.13 23.13 

Figure 4 presents 12 profiles with four patterns much like those 
in the previous figure, except that all have the same average level. 
These 4 modal profiles were correctly identified and the subtest 
score values obtained for each were: 

L 1.87 1.87 8.87 8.87 
2. 5.13 5.13 6.13 6.13 
3. 8.87 1.87 8.87 1.87 
4, 6.13 5.13 6.13 5,18 

It may never be possible to match all profiles to a few modal pat- 
terns, but the percentage of indeterminate cases might be reduced 
by identifying which tests are most important in the modal pat- 
terns. That is, the reciprocal of the average D? for each subtest of 
the profiles in the original cluster can provide weights to be used to 
establish relationships between undiagnosed Ss in the sample and 
the modal patterns. When near zero weights are assigned to some 
subtests by the program, the comparison is reduced to using the re- 
maining subtests for diagnostic signs rather than modal patterns 
(profiles). 

It seems desirable to mention that all modal patterns must be 
cross-validated before they can be presumed to have diagnostic 
value, since random sampling error may have contributed greatly 
to their formation. Also, different patient samples used in cross- 
validating may disclose additional patterns not sampled earlier. 
Features irrelevant to diagnosis, such as sex and education, also 
may cause a systematic separation of modal patterns, and it 
may prove necessary to propose different modal patterns associated 
with psychiatric disorders for different classes of people. 

No systematic proposal for determining the fit of a S to modal 
patterns is made here. It is supposed that visual inspection and 
comparison of profile and patterns will suffice for clinical purposes. 
For more quantitative matching of a profile to patterns the simple 
statistics of du Mas (1963) would be appropriate, 
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Aw exact analysis of variance F-test can be obtained through a 
permutation procedure due to Fisher (1935), and from a theoretical 
point of view the normal theory F-test is an approximation to this 
exact test. Although a large number of computer programs exist for 
the usual normal-theory analysis of variance and its associated 
F-tests, only a single special purpose program has been reported for 
the F-test under permutation (Baker and Collier, 1961). The pres- 
ent paper describes a general purpose program for obtaining the 
empirical distributions of variance ratios under permutation for a 
wide class of experimental designs. In order to clarify the nature of 
this computer program, a brief description of the permutation pro- 
cedure is presented in the paragraph below. 

Under the permutation procedure the distribution of a particular 
variance ratio, say Fs, can be produced from just the data collected 
in a single experiment. The distribution is obtained by permuting 
the sample data in accordance with the constraints imposed by the 
experimental design and by computing F; for that permutation pat- 
tern. The data are then repermuted, F; recalculated, and the proc- 
ess repeated until a sufficient number of Fps have been obtained 
to form the empirical distribution of F;. One then defines a critical 
region on this distribution, say the upper five percent of the cases, 
and if the value of F; yielded by the sample data falls in this region, 
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the corresponding null hypothesis is rejected. There are two ad- 
vantages to using the F-test under permutation, One, only the data 
at hand are necessary to form an exact test of the null hypothesis. 
Two, there are no underlying assumptions of normality such as are 
required by the usual analysis of variance procedures. For addi- 
tional discussion of the permutation approach, the reader is referred 
to Scheffé (1959, Chapter 9) and Kempthorne (1952). 


The Program 


In order to produce an efficient computer program for the above 
Procedure, the program was broken into sections A, B, and C which 
are under control of a higher level executive routine. Section B 
permutes the data, performs the analysis of variance, and yields the 
empirical F’s, Section A was assigned a number of tasks which 
adapt the internal configuration of section B to the particular data 
and analysis employed. These tasks are performed only once; 
hence were programmed as a separate section. Section C is a post- 
processing routine which arranges the obtained F’s into frequency 
distributions and produces the output listings, Generality in the 
analysis of variance (ANOVA) aspects of the program was ob- 
tained by using a general purpose ANOVA program due to Jennrick 
(1961), which can handle any balanced fully replicated or nested 
design with up to eight factors involving an equal number of ob- 
servations per cell of the design. In order to Provide complete gen- 
erality, it was necessary to design a permutation program with 
generality matching that of the ANOVA program. The permutation 
scheme consists of two Programs, one which translates a set of 
codes specifying the Permutation to be performed (DECODE) and 
one which actually permutes the data (PERMUTE). The operation 
codes implemented were as follows: 

* —-signifies blocking. The array is separated into blocks, each 

of which is processed separately. 

$ —means permute over. The observations to the left of the $ 
are permuted over the cells on the right. 

( )—signifies a set. The letter within the parentheses defines the 
size of the set. A set is a group of observations to be per- 
muted as a logical entity. 

++ —signifies then. Used to Separate successive stages of permu- 
tation. 
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e —means end of specifications. 

For purposes of illustration, let us define an observation as 
Xur and employ a randomized block design with I blocks, J treat- 
ments, and K observations per cell. The permutation procedures re- 
quired by this design are as follows: 

1. The JK observations within a given block be permuted. 

2. This process be repeated separately for each block in the de- 

sign. 

Such a procedure can be specified using the operation codes by 
JK$JK*I., which means permute the JK observations into JK cells 
and repeat this for I blocks. In the case of a design which involves 
two stages of randomization, let us define an observation as Xiu 
with I replications, J blocks, K treatments, and L observations per 
cell. A permutation specification could be KL$KL*IJ + (L)$JK*L., 
in which case the KL observations are permuted into KL cells and 
this is done for IJ blocks. The second stage consists of treating the 
sets of L observations as a logical unit and of permuting the sets of 
size L into JK cells for each of I blocks. Given a permutation speci- 
fication such as one of the above, the subroutine DECODE uses the 
numerical values corresponding to the levels of the index letters to 
compute the locations of the corresponding arrays, as well as em- 
ploys the operation codes to establish the permutation to be per- 
formed. The DECODE routine then sets indices, initializes arrays, 
and prestores the routine PERMUTE which actually does the per- 
muting of the data. Under this scheme the decoding operation is 
performed only once in section A. Thereafter, the permute routine 
properly manipulates the data. 

The separation of the specification or prestore stage from the com- 
putational stage is also maintained in the ANOVA programs. A 
subroutine named ANOVA-A in section A reads in all the informa- 
tion related to specification of the analysis of variance, the error 
terms to be used, and the variance ratios to be computed. The in- 
formation generated from these specifications is used to prestore 
the program ANOVA-B in section B which computes the variance 
ratios desired. Thus, once the configurations of the PERMUTE and 
ANOVA-B routines have been established, they merely permute, 
compute, and store the F’s until the desired number of values have 


` been created. 


172 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Output 


When the proper number of F’s have been computed and stored, 
section B terminates and the post-processing routines of section C 
are entered. Section C reads in a set of cards specifying the output 
desired and then processes the F’s provided by section B in accord- 
ance with these specifications, The output from section C consists 
of the following information for each F-distribution specified: 

1. A grouped frequency distribution of the empirical F’s with the 
midpoints of the intervals being the 9 decile points and P.s, 
P.ors, and P.ss. The normal theory F’s corresponding to these 
points are also given. 

2. The cumulative proportion from the upper end of the empirical 
distribution to the lower end. 

3. The average error mean square for each variance ratio. 

In addition, the complete ANOVA table for the original sample 
data is presented as are the original data. 


Capabilities and Limitations of the Program 


There are a large number of specific “do’s” and “don’t’s” given in 
the program manual and those presented below are given only to 
Provide the reader with some sense of the scope of the program, 

. Randomization patterns and ANOVA can be specified for a 
single variable Possessing up to eight subscripts. 

Up to 20 stages of randomization can be specified, 

The product of the number of levels of the indices must be less 
than 2500. 

4, The number of variance ratios specified cannot exceed 20. 

5 


m 


w w 


cannot exceed 20. 

The number of samples generated within a computer run is = 

1000, 

7. An option exists whereby non-null treatment effects can be 
added to the permuted data prior to Computing the F's, Thus, 
one can obtain the power of the F-test under Permutation for 
the non-null treatment effects applied. 

8. The program written in Control Data FORTRAN 63 con- 
tains two short machine language subroutines, 

9. Systematic accounts as to running times have not been main- 


> 


> 
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tained, but for small designs the times are quite short. For ex- 
ample, a randomized block design, three blocks, three treat- 
ments, and six observations per cell, requires four minutes for a 
run of 1000 permutation patterns. 


Summary 


The permutation model devised by Fisher is yet another example 
of a powerful statistical technique which has been avoided because 
of its computational labor. Since the digital computer has removed 
nearly all restrictions imposed by computational labor, techniques 
such as the permutation procedure should become an integral part 
of the statistical tools available to the educational and psychologi- 
cal research worker. 
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A SET OF SOCIOMETRIC FORTRAN II ROUTINES! 


LEOPOLD 0. WALDER 
University of Maryland 


Tun processing and analysis of peer-rating data represents an 
ideal use of computers. This work when done by human clerks not 
only is expensive in time and boredom but also is full of error. The 
present paper is an updating of a report of a system devised for the 
IBM 650 (Walder, Greene and Lefkowitz, 1962). This set of routines 
for the IBM 7094 does more than a system of programs (see also 
Toigo, 1962) for the 650 did; this is the result of the greater 
capacity of the 7094. This sociometric package was written (coded)? 
in FORTRAN II for the IBM 7094 with an off-line IBM 1401 
used as a card-to-tape converter and as an off-line printer. 

A set of 11 subroutines was written to perform various functions, 
A main program (without the DIMENSION and COMMON state- 
ments) which calls all the subroutines is listed here: 

CALL RDNAJU 
1 CALL ZEROMX 
CALL ZSUMS 
CALL RDJCDS 
CALL SUMMX 
CALL PERCNT 
CALL PRINMX 


1Computer time given by the Computer Science Center of the University 
of Maryland is gratefully acknowledged here. 

2A source program listing which includes operating instructions has been 
deposited as Document number 8625 with the ADI Auxiliary Publications 
Project, Photoduplication Service, Library of Congress, Washington, D. C. A 
copy may be secured by remitting $1.25 for 35 mm. microfilm or $1.25 for 
photoprints. 
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CALL CHOICE 
CALL CHOSEN 

CALL RECIPR 

CALL IGNORE 

GO TO 1 

END 

SUBROUTINE RDNAJU. This subroutine reads the cards con- 
taining the names and 2-digit numbers of the members of the 
group into arrays used later for the output from subroutines 
SUMMX, PRINMX, CHOICE, CHOSEN, RECIPR, and 
IGNORE. 

SUBROUTINE ZEROMX. The matrix elements which store the 
choice (1) or non-choice (0) of the i-th Judge concerning the j-th 
object are cleared to zero. This subroutine could be bypassed if two 
or more matrices were to be summed. 

SUBROUTINE ZSUMS. The row sums of the judge scores and 
the column sums of the object scores are cleared to zero. This sub- 
routine could be bypassed if these scores were to represent sums 
from more than one item matrix, 

SUBROUTINE RDJCDS, This subroutine reads cards contain- 
ing the item wording and then judge cards (each containing the 
number of the judge, the item number, and the numbers of those 
group members he nominated). A one is added to the matrix ele- 
ment corresponding to the i-th row (number of the judge) and to the 
j-th column (number of the group member nominated). The item 
number is monitored to avoid mixing data from several items. 

SUBROUTINE SUMMX. Column (object) sums and row 
(judge) sums are calculated. The row and the column sums are cal- 
culated and printed, labelled with the names and numbers of the 
group members. 

SUBROUTINE PERCNT. For each sum a corresponding per- 


(row sum) / (number of objects X number of items). 
SUBROUTINE PRINMX. The matrix is printed. At the far left, 
the rows are labelled with the names and numbers of the group 
members; at the top, the columns are labelled with the numbers of 
the group members. 
SUBROUTINE CHOICE, This prints the names of those objects 
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chosen by this judge. (This reproduces the response sheet of each 
judge). 

SUBROUTINE CHOSEN. This prints the names of those judges 
choosing this object. 

SUBROUTINE RECIPR. This prints the names of those who 
reciprocated choices. 

SUBROUTINE IGNORE. This prints the names of those who 
reciprocated non-choices. 

This system yields the output needed to detect a classroom’s 
choice pattern. At present the author is developing a response sheet 
acceptable to the IBM 1232 optical reader which will bypass the 
manual key-punching of cards. 
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A FORTRAN ITEM ANALYSIS PROGRAM FOR ITEMS 
SCORED ON A CATEGORICAL OR INTERVAL BASIS* 


RICHARD L. GORSUCH 
Vanderbilt University 


Psycnonocicat and achievement tests are usually scored by 
either of two methods: (1) categorical scoring where one and only 
one of several possible answers is given credit as “correct,” or (2) 
interval scoring where each of the different possible answers to the 
item receives varying degrees of credit. An example of the former is 
the multiple choice achievement test. Of the four or five possible an- 
swers the individual may select, one is scored as correct and all 
others receive no credit. An example of the latter, interval scoring, 
is the 16 PF (Cattell and Eber, 1964). On this personality test, a per- 
son can choose any one of three answers and one extreme answer 
receives a score of zero, the middle answer a score of 1, and the 
other extreme answer a score of 2. Another example of categorical 
scoring is found in the semantic differential (Osgood, Suci, and Tan- 
nenbaum, 1957); on each item, the person marks any one of several 
different positions on a line and receives a score according to how 
far his “x” is from a given end of the line. 

Whereas programs for various computers are available to analyze 
categorically scored items (Baker, 1963; Dick and Spencer, 1965; 
Jaspen, 1962; Jones, Pullias, and Michael, 1965; Nichols and Tetz- 
laff, 1965; Saunders, Gaddis, and Michael, 1962), none appear to be 
readily available for items scored on an interval basis. Furthermore, 
few programs make provisions for one or more external criteria be- 


1The development of the program presented in this paper was supported 
by a grant (HD947) to C. D. Spielberger from the National Institute of Child 
Health and Human Development, United States Public Health Service. 


179 


180 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


ing included in the item analysis program even though this is often 
desirable. Therefore, it was necessary to develop a FORTRAN pro- 
gram for the IBM 7072 which would score and item analyze 
tests where the items are scored by either a categorical or an in- 
terval method. Provisions for evaluating items against external 
criteria were also included. 


Characteristics of the Program 


The program was written in basic FORTRAN for the Vanderbilt 
University IBM 7072 which is fed by an IBM 1401 satellite com- 
puter. The IBM 1401 converts the data from cards to magnetic tapes 
for input and processes both punched and printed output. The 
Program was constructed to run under the Vanderbilt Automatic 
Monitoring Operating System, but it can be adapted to almost any 
FORTRAN system. (Source deck listings are available from the 
author.) 

For each item being evaluated, the program calculates the percent 
of the sample who use each of the possible responses to that item. 


of the item with Scores on each external variable is also calculated. 
The product-moment formula, used for both external and internal 
evaluation, gives phi, Point-biserial, Point-polyserial (Jaspen, 1946, 
1965), or the usual Pearson product-moment correlation depending 
on the nature of the data. Tf both item and criterion are categorical, 
the result is a phi Coefficient. If the item is categorical and the 
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criterion is continuous, the result is a point-biserial coefficient. 
The coefficient is of the point-polyserial family when the item has 
several scoring itervals but the criterion is continuous. All of these 
coefficients are estimates of the usual Pearson product-moment co- 
efficient without the assumption of a normal distribution of scores 
of the parent population on the continuum underlying the cate- 
gories. 

The program presents the means and standard deviations of the 
test scores and of the criteria scores. The internal consistency re- 
liability of the test is calculated by the alpha coefficient (Cronbach, 
1951) which is KR-20 in the special case of the categorical items. 
The program also prints out the number of subjects on which the 
item analysis is based and a job identification. If desired, each per- 
son’s raw score will be punched into a card which also contains up 
to 10 columns of identifying material from his data card. 

The input is presently limited to tests having a maximum of 250 
items where each item response is read in as a one-digit character 
(i.e., item scores can only vary from 0 through 9). Up to five 
criterion variables may also be included for item evaluation. The 
number of subjects is unrestricted by the program. 


Preparation of the Data 


A control card follows the program deck with its header cards 
(which give the Vanderbilt monitor system the user’s budget number 
and other essential information). The control card is punched on an 
IBM 80 column card as follows: 


Columns Entry 

1-3 Number of items in the test (up to a maximum 
of 250) 

6 1 for categorical scoring, 2 for interval scoring 

9 The minimum score any subject can receive on 
an item (0 for categorical scoring) 

12 The maximum score any subject can receive on 
an item (1 for categorical scoring) 

15 Number of other variables with which each item 
is to be correlated (up to a maximum of five) 

18 1 if punched scores for each subject are not de- 
sired (left blank if punched scores are desired) 

21-80 Job identification (determined by user) 


Following the control card is a card or cards containing the 
scoring key for the particular test being scored. As many cards as 
necessary are used. The first scoring key card contains the scoring 
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instructions for items 1 through 80, the second for 81 through 160, 
and so on. The scoring key cards allow the data to be punched with- 
out regard to the direction of scoring of individual items. Beginning 
in column one, a number is punched for each item in consecutive 
columns of the card(s). If the scoring is categorical, each column 
is punched with that category (0 to 9) which indicates the correct 
answer for that particular item. If interval scoring is used, each col- 
umn is punched with a 0 unless the scoring is to be reversed. If the 
scoring for a particular item is to be reversed from the way in which 
it was punched, a 1 is put in the appropriate column of the scoring 
key card. For example, on a Likert attitude scale a person may mark 
any number from 1 to 5 to indicate the degree of his agreement 
with an attitude statement, which may be positively or negatively 
related to the attitude being measured. Let us assume that the first 
column of the scoring key card contains a 0 and the second a 1 to 
indicate the direction of scoring. Complete agreement with item 1 
indicated by a 5 on the person’s data card, would then add five 
points to his total score. Complete agreement with item 2, also in- 
dicated by a 5 on the data card would, however, indicate a response 
against the attitude being measured. The scoring would be reversed 
and therefore only one point would be added to his total score. 

The next card after the scoring key cards contains the 
FORTRAN format statement describing each person’s data. The 
format begins with a double A field, but all data fields are in float- 
ing point notation. For example, the following format might be used 
for a 100 item test with three criterion variables: (4X, 243, 10X, 
50F1.0/20X, 50F 1.0, 3F5.2). The data cards then follow. Each per- 
son’s criterion scores are punched after that person’s item scores. Af- 
ter one person’s data are read, the second A field is checked by the 
Program; if it is blank, the Program assumes that the last person’s 
data have already been read, Therefore, the last subject is followed 
by as many blank cards as are required for one observation to tell 
the program that the last set of item responses has been read. 

Any number of sets of data can be scored on the same pass by 
Tepeating the above Sequence of control card, scoring key card (s), 
format card, data cards, and blank end-of-the-data card(s). A 
blank card after the blank end-of-the-data card(s) indicates to 
the program that the last set of data has been processed and the run 
is completed. 
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SCORING TEST BATTERY: 
A PROGRAM FOR THE IBM 7094 


ARIEH LEWY an» WILLIAM CRAWFORD 
University of Illinois College of Medicine 


Tus program is designed to score a complex test battery and to 
provide individual scores, test statistics, and item data for each sub- 
test separately and for the battery as a whole. It may also be used 
for scoring a test which is not divided into subtests. Both printed 
and punched output are produced. 

A great saving in computer-time is accomplished by scoring all 
items in one pass through the computer after which item scores are 
assigned to subtest categories. Therefore it is not necessary to search 
core storage repeatedly or to rewind tapes to obtain scores for each 
subtest. 


I. Restrictions 


The program is designed to score a test composed of a maximum 
of 500 items and 26 subtests. No item may be assigned to more 
than three subtests. The number of alternatives may range from two 
to nine and need not be the same for every item. There may be no 
more than 20 item response cards per subject (S). 


II. Options 
Subtest classification 


Each item may be assigned to as many as three subtests. This 
feature permits cross-classification of items so that any item may 
contribute to more than one subtest score. A separate score is com- 
puted for each subtest, and the total score is accumulated by count- 
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ing each item only once regardless of the number of subtests to 
which is contributes, 


Weighting 


Each item may be assigned a different weight, but the weight for 
an item must be the same in every subtest to which it contributes, 


posite weighted total score, Both options can be used simultaneously 
to obtain subtest scores based on weighted items and a total score 
based on weighted scores and weighted items, 
T Scores c 

In addition to the raw and percent scores produced as standard 
output, 7’-Scores may be obtained for each of the subtests and for 
the total test. 


Card Sequence Control 


This option may be used to compare identification numbers only, 
or to check both identification numbers and card sequence for each 
S. The sequence control statements compare identification numbers 


interruption. t 

The card numbers must be positive integers in increasing order 
for each § Separately, However, the integers need not be consecutive 
nor need they be the same for each S. Therefore sequence control 
can be used when only certain cards for each S are selected for 


special Processing, 
III. Input and Output 


Input is in the form of punched cards which contain an § identifi- 
cation number, card sequence control numbers, and raw item Te 
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sponses. There may be as many as 20 cards per S, and each card 
may contain up to 73 item responses, provided that the total num- 
ber of responses per S does not exceed 500. One variable format 
statement governs the reading of all data cards. This statement 
must be prepared so that it continues over as many cards as there 
are data cards for each S. 


Printed Output: 


(1) Individual Scores 
(a) raw scores for the total and each subtest (standard), 
(b) percent scores for the total and each subtest (stand- 
ard), 
(c) weighted scores for the total and each subtest (op- 
tional), * 
(d) T-scores for the total and each subtest (optional) ; 
(2) A frequency distribution of scores on the total test and on 
each of the subtests; 
(3) A complete intercorrelgtion matrix for all scoring categories; 
(4) The proportion of Ss selecting each alternative for every 


item; 
(5) Item statistics computed with respect to the total and every 
subtest: 
(a) difficult, 
(b) item standard deviation, ' 


(c) point biserial correlation, 
(d) index of reliability; and 
(6) The following test statistics for each subtest and the total: 
(a) maximum possible score, 
(b) mean and its standard error, 
(c) standard deviation and its standard error, 
(d) skewness and its standard error, 
(e) kurtosis and its standard error, 
(f) reliability (KR-20) 


The intercorrelation matrix and all subtest and total scores may 
be obtained as punched output. Punching is controlled by a vari- 
able format statement so that the output may be obtained in a 
format which can be used immediately for research or administra- 
tive purposes. 
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IV. Computing Procedures 


Items are assigned to subtests by a vector of two-digit fixed point 
numbers which indicates the subtest number of items appearing on 
corresponding columns of the data cards, 

Item weights are assigned by a vector of two-digit floating point 
numbers. When items are differentially weighted both the total 
Score and the subtest scores are influenced by the weighting factors 
for each item. 


V. Computing Time 


All items are scored in one pass through the computer before 
item scores are assigned to the total and to subtest categories. The 
program is very economical, since it is not necessary to waste 
valuable computer time by repeatedly searching core storage or 
rewinding tapes to obtain scores for each subtest, For example, a 
test consisting of 361 items classified into 15 subtests taken by 196 
students was scored in 2.96 minutes, 


Summary 


A rapid and efficient program for the IBM 7094 has been de- 
veloped which scores and provides complete item and test data for a 
complex battery of multiple choice tests. It also provides options for 
card sequence control, item weighting, and score weighting. Input 
is in the form of punched cards containing student identification 
numbers, card sequence control digits, and raw item responses, Out- 
put consists of punched and printed scores and of printed item 
and test data, 


X 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
Vor. XXVI, No. 1, 1966 


A 7094 FORTRAN PROGRAM FOR THE COMPUTATION 
OF TETRACHORIC CORRELATIONS! 


STEVEN G. GOLDSTEIN, JAMES D. LINDEN 
anp DAVID A. STUDEBAKER 
Purdue University 


A variety of computer programs in use today depends upon cor- 
relation matrices for the solution of problems. Many of the original 
data utilized in the construction of these matrices are in dichoto- 
mous format. Because of the unavailability of a program for the 
computation of the tetrachoric correlation, Pearson product-mo- 
ment correlations generally have been employed which provide 
spurious estimates of relationships among the data involved. As a 
consequence of the need for a program that would compute tetra- 
choric correlations to provide more appropriate item reliability in- 
formation and appropriate input for factor analytic work, the 
authors developed the program reported here, i 


Characteristics of the Program 


Preceding a listing of the tetrachoric correlation matrix, the 
program listing provides each subject’s total score and a frequency 
distribution of such scores along with the mean, median, variance, 
and standard deviation of the distribution. The analysis utilizes a 
two-by-two contingency table peculiar to each set of dichotomous 
variates represented in the resultant matrix. All possible off diago- 
nal variate combinations are considered. In each case, an ad/be 
Tatio is calculated as described by Downie and Heath (1959). This 


1The development of this program was supported in part by Contract No. 
N62269-2670, Naval Air Development Center, Johnsville, Pa., Principal In- 
vestigator, Donald R. Brown. 
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resultant ratio is compared to the estimates of values for the 
tetrachoric correlation statistic determined by Davidoff and Goheen 
(1953), and each value selected is either listed or both listed and 
punched in a variable format. One option arbitrarily provides unity 
values on the main diagonal of the matrix. Another option substi- 
tutes a high-low median split of the total subject scores for one 
dimension of the contingency table and provides main diagonal 
tetrachoric estimates of the correlation of a given variate with the 
total score. 

Because the Scanning and storing operations demanded by the 
Program may exceed the storage capacity of ths system, the pro- 
gram provides for the temporary storage of information on tape 
until the monitor calls forth the appropriate information for the 
output tape. 

This program is written to be run under the IBSYS monitor, 


Job Deck Set-Up 


First Card: This is generally the system identification card and 
is peculiar to each individual user’s system. The 

‘ program deck immediately follows this card, 
Second Card: (Title Card) Any title desired may be punched into 
columns- 1-72 and is printed out at the top of each 


listing page. 
Third Card: (Problem Card) 
jt Columns 1-3. The number of subjects (less than 
P 171). 


Columns 4-6. The number of items or dichotomous 
variates (less than 121). 

Column 7. A “o” punch arbitrarily yields unity 
values along the main diagonal of the matrix. A 
“1” punch substitutes the correlation of each item 
with the total score for main diagonal unity values. 
Column 8. A “0” punch will list the contingency 
Tatio resultants. A “1” punch ignores such listing. 
Column 9. Punched BCD output is obtained with a 
“0” punch, whereas a “1” punch provides only the 
tetrachoric correlation listings, 

Column 10. The number of data input format cards. 
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Columns 11, 12. The number of correlations per 
BCD card (less than 12). 

Fourth Card: (Data Input Format Cards) The program demands 
an identification number for each subject on the 
first card of each set of subject cards. Thus the 
format card must begin with an integer identifica- 
tion number. Subsequently, any number of columns 
may be omitted and data read as desired. The 
entire card should be in Integer format. 

Data Cards: Data cards follow immediately. They are set up in 
such a way that each subject set is complete before 
the next subject set begins. Data must be punched 
either “1” or “0.” They may extend for the entire 
card field. 


Punched Output Cards 


These cards are identified by the matrix row and then by the 
ordinal position of the card in the make-up of the row. The output 
which is in Floating Point format (F5.2) is fixed by the program. 
The punches in columns 11 and 12 of the problem card indicate the 
number of correlations contained on the card. The deck of such BCD 
cards may then be used as an input matrix for a factor analysis or 
for any program that requires correlation matrix input. 
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A PROGRAM FOR COMPUTING CANONICAL CORRELA- 
TIONS ON IBM 1620! 


E. ROSKAM 
Eindhoven Institute of Technology, Netherlands 


Introduction 


CaxonrcaL correlations are called for (e.g.) when relationships are 
investigated between a set of predictor variables and a criterion 
consisting of a number of variables that cannot on a priori grounds 
be combined into a single measure. Partial regression weights can be 
found which maximize the correlation between the weighted sum of 
predictors and the weighted sum of criteria, Besides these, different 
sets of weights can be found providing independent weighted sums, 
thus giving 2nd, 3d ete, canonical correlations. The maximum 
number of independent sets of weights is equal to the number of 
predictors or to the number of criteria, whichever is the smaller. 
Each set of weights can present valuable information concerning 
the construct validity of each of the variables separately. It is 
therefore worthwhile not only to compute the maximum canonical 
correlation, but also to proceed to the smaller ones, as long as they 
reach statistical significance. 

Two FORTRAN programs for computing canonical correlations 
have been described by Cooley and Lohnes (1962). One of these 
computes the maximum canonical correlation only and it does not 
assess its statistical significance. The other program computes all 
canonical correlations and also computes chi-square tests for each 
of them. 


1The program has been tried out successfully on the IBM-1620 in the 
Mathematical Department of the Eindhoven Institute of Technology. I am 
indebted to Dr. A. J. Geurts for his kind cooperation and assistance, 
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For many practical purposes the latter program performs too 
much, because usually only the first few canonical correlations will _ 
be statistically significant. Moreover, its length is prohibitive for _ 
use on a small computer. i 

This paper describes an alternative program, which computes 
canonical correlations in decreasing order of magnitude and pro- 
vides statistical tests for each. The number of canonical correlations 
to be computed is given as input, but it might also be decided by 
the machine on the basis of a given level of significance, 


Rationale 


Input to the computer is a super-matrix of correlation coefficients 
between m, predictor variables and Me criterion variables. 


J Ryp Rye 
intercorrelations intercorrelations 
among predictors between predictors 
and criteria 
Re Re 
transpose of Ry. intercorrelations 
among criteria 
Since it is symmetric, the lower triangular half can be omitted. 
In the canonical correlations model it is irrelevant which set of 
variables is considered as predictors. The following mathematical 
expressions are valid only if the smaller set is considered as criteria. 
The i-th canonical correlation (r:) is the square root of the i-th 
latent root of the matrix 


x — RRR RS, (69) 
The weights (wa) associated with the criteria are the elements 
of the corresponding latent column vector normalized to unit length. 
The weights for the predictors are given by the vector 


Woi = Ro Rp Wes/ri, @) 
normalized to unit length, a 


The statistical test of significance for the i-th canonical correla- 
tion is given by 


T: = —[N — i(m, +m, + 1)] p3 log, (1 =r) T <7 (3) 
izi m. < m, 

where N is the sample size. This is distributed as chi-square for 

(my —i+1) (m —i+1) degrees of freedom, For details and math- — 
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ematical proofs of (1), (2) and (3), see Cooley and Lohnes 
(1962), Anderson (1958), and Bartlett (1941). 

The computation of the chi-square test seems to involve the com- 
putation of all canonical correlations. However, a method which 
greatly simplifies this labor has been found. For i=1, the sum in 
(3) is equal to the natural logarithm of the determinant? of I — X. 
For i=2 it suffices to subtract log,(1—ri2) from this sum to calcu- 
late the chi-square test for r2, etc. This method permits of testing 
the significance of each canonical correlation as it is computed in 
decreasing order of magnitude, without computing all of them sepa- 
rately. 

Two requirements are essential to use this advantageous possibil- 
ity: (a) The availability of a method for assessing the determinant 
of a matrix without unduly lengthening the computational program; 
in fact a subprogram for matrix inversion (which is required any- 
how) was used which provides the determinant of a matrix as a 
byproduct.’ (b) The use of an extraction method which provides the 
latent roots of (1) successively in decreasing order of magnitude. 

One method of computing the largest latent root and vector, 
which is adequate in most cases, is the standard iterative method 
(also known as the Hotelling method). Since matrix (1) is not sym- 
metric, it is not possible to compute the next largest latent root and 
vector by means of the extraction or’ exhaustion method in the usual 
wayt. 

Anderson (1958), however, describes a relatively simple proce- 
dure which permits of the consecutive application of the iterative 
method to subsequent residual matrices. This method consists of 
first normalizing the criterion weights in such a way that 


w.,*/R.Wes* = 1 (4) 
where * denotes the normalized vector. Let now W,* stand for the 
matrix of normalized criterion weights associated with all canonical 
correlations. 


2I am indebted to Prof. G. W. Veltkamp for pointing out this relationship. 
Proof: Let à be a latent root of X. Then |K—AI|=0; |—X+Al[=0; 
|I—X—I+a1|=0; |(I—X)—(—a)1|=0; hence 1—) is a latent root of I—X. 
Therefore |I—X|=0(1—)), Q.E.D. 

8 This subprogram is part of the standard software of the Mathematical 
Department of the Eindhoven Institute of Technology, where it has been 
developed and tested. Particulars of this subprogram are given below. 

Unless by also computing the latent row vector, which is unduly laborious. 
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therefor: 


W.*’R..W.* = I 
or 


W.*’R.. = W (5) 
Hence, for the i-th canonical correlation 


W.."’R,, = W.,*” (6) 
Th this expression Wa” is the i-th row in W,*+. Anderson shows that 
the largest latent root of 


Xin = Xi rw R, X =X (7) 
is equal to the second largest latent root of X;. Hence the (i-++-1)th 
canonical correlation can be found by applying the iterative method 
to Xi44. 

An obvious advantage of the iterative method is that it requires 
a relatively short program, and, secondly, that the computation of 
canonical correlations can be stopped after a predetermined num- 
ber or if the next r were below a certain level of statistical signifi- 
cance. Other methods of computing latent roots and vectors of non- 


more nearly equal latent roots, In most cases, however, only the first 
few latent roots, which usually are well Separated, will provide sig- 
nificant canonical correlations, and therefor this disadvantage is 
not very serious, 


Computational Flow 


The above procedure has been programmed in FORTRAN for 
IBM 1620 (memory storage 6000 words). The program is adapted 


The computational flow, omitting details, is as follows: 
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The subprogram Inverse operates on a matrix M stored in A. This 
subprogram is composed of two parts. Inverse part one replaces the 
matrix in A by an uppertriangular and a lowertriangular matrix, 
so that LU=M. The diagonal elements of U are unity and they are 
not stored. 

Next Inverse part two replaces the two triangular matrices by 
the inverse of M. 

Inverse part one is used for the computation of the determinant 
of M. As the determinant of U is unity, the determinant of L will 
be equal to the determinant of M. Since L is triangular, its de- 
terminant is equal to the product of its diagonal elements. 

The main program consists of the following steps: 

1. R, and R,, are read into the two-dimensional arrays A en C 

respectively ; 
2. Rpp (located in A) is subjected to the subprogram Inverse. 
A now contains Rpp. 

3. The product R,,1R,, is computed and stored in the two- 
dimensional array P. 

4. Re is read into both A and the two-dimensional array D. A is 
subjected to the subprogram Inverse so that is replaced by 
Re, 

5. The product X = Re Rep (Rpp*Rgo) is computed and stored 
in A. It will be noted that Rpp *Rpo is preserved in P for the 
computation of the predictor weights; Roo is preserved in D 
for the computation of the residual matrix X;1. 

. X is copied into C. I—X is placed into A and subjected to 
Inverse part one, and log.|I—X| is computed; this value is 
referred to as PI. 

. The largest latent root and vector of the matrix stored in C 

(consecutively Xı,Xs, etc.) is computed by means of the 

iterative method. Canonical r, criterion and predictor weights 

and chi-square are computed and printed. The program either 
comes to an end if the required number of canonical correla- 
tions is reached, or it transfers to the next step: 

PI is decreased by loge(1—r?) for the chi-square test of sub- 

sequent canonical correlations. 

9. The criterion-weights are further normalized by dividing 

them by the scalar Vw,’ RecWe. The normalized vector is mul- 


tiplied by Re to provide wo*. 


oa 


a 


go 
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10. The residual matrix X,,, = X; — TW Wa” is compu 
and stored in C. The program transfers back to step No. 7. 


The program provides for an automatic STOP in the rare cases 
that a canonical correlation should equal unity or if one of the 
inverses does not exist. A.STOP is also provided in case the compu- 
tation of the latent root and vector should not converge. 

A complete FORTRAN listing can be obtained from the author, 
or from the Director of the Testing Bureau, University of Southern 
California, Los Angeles, California. 
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TO ERR IS INHUMAN: EFFECTS OF A COMPUTER 
CHARACTERISTIC 


KENNETH I. HOWARD anv ROBERT W. LISSITZ+ 
The Institute for Juvenile Research 


Hicx speed computer equipment is in very wide use today, and is 
playing a particularly important role in the behavioral sciences. 
With the introduction of low rental “desk-size” computers, smaller 
research establishments are able to support computer installations. 
However, computer experts are not always available to these in- 
stallations for general consultation, and the individual researcher 
frequently finds it more convenient to write and operate his own 
programs. This tendency has become more prominent with the pro- 
liferation of simplified programing languages. 


Truncation Error 


Reliance on computers, often to the neglect of basic research 
techniques (e.g., random groups, reliable rating scales), has been 
criticized frequently (for example, see the “comment” in the Ameri- 
can Psychologist, 1964, by Crumbaugh). Most of these criticisms, 
which have focused on the quality of data collection and prepara- 
tion for calculations, emphasize that confidence in the results of a 
project should not be a function of the complexity of the data proc- 
essing equipment employed. One aspect of machine use which is not 
generally mentioned in these criticisms, however, is that of the po- 
tential effects of “truncation error.”? Truncation may cause gross 


1 Now at Syracuse University. 

2 Truncation is the loss of digits imposed by the computer to keep a con- 
stant size mantissa. For example, if .15992 is multiplied by .12613, the result 
is 0201707096. When this problem is handled inside the computer (as distinct 
from a desk calculator) the result would be truncated to 20170709 and an 
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calculation errors. It derives from a characteristic of the compute 
system, not from characteristics of the research. 

Neely (1965) has done the basic work on the internal documen- 
tation of computer programs at the University of Chicago Biological 
Sciences Computation Center. He has demonstrated marked effects 
of truncation error and is presently completing a full-length presen: 4 
tation for computer users and programmers. The following com- ` 
ments and illustrations, which bring one aspect of Neely’s conclu- 
sions to the attention of behavioral scientists, are provided as a 
caution, particularly for the relatively unsophisticated computer 
user, ; 

Most fixed word length computer programming systems (the most | 
common type) use a fixed mantissa of eight digits and an exponent 
(with a base 10) from —99 to +99. Because of truncation the num- 
ber represented inside the machine as the result of a mathematical — 
calculation is always less than or equal to the actual number. There- _ 
fore, whenever a datum is greater than eight digits at input time, or 
whenever the result of a calculation is a number with more than — 
eight digits (and the integers past the eighth place are not zeros), i 
an error has occurred and these errors are cumulative throughout a - 
problem. j 

It is evident that truncation error is a function of the number of — 
significant digits, but there is an important implication which is nob _ 
80 obvious—different formulae (algebraicly equivalent) for the 
same statistic (using different steps to arrive at the same end) will _ 
give different values for the same data set, 


Demonstration of Effects of Error 


In order to demonstrate this Point, a program was written which _ 
would calculate the mean, variance, and product moment correla- _ 
tion, each by two formulae, These formulae are shown in Tables 1 


adjustment in the exponent would also take place. First, the computer has the _ 
bapa te E K ve and the number .12613000 X 10°, and then performs 

e multiplication by referring to th lace i i te 
is 20170709 se tae Tring e proper place in the table and the resul y 
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and 2. Data were constructed which would most clearly illustrate the 
effects of truncation error. Seven problems were run for each for- 
mula. Each problem had an N of 100, but a different number of sig- 
nificant digits. In each of the seven sets, the numbers were in ascend- 
ing order in terms of the last three digits. 


TABLE 1 
Mean and Variance Using Two Formulae for Each 


Variance 
2y 2X X- w3wxX—X? NX- (2x) 


Range of — — + = 


N Ne 


. 1-100 50.5 50.5 833.25 833.25 
TI. 901-1000 950.5 950.5 833.25 833.25 
. 9001-9100 9050.5 9050.5 833.25 796.00 
. 90001-90100 90050.5 90050. 5 833.25 —2400.00 
. 900001-900100 900050.5 900050.5 833.25 —350000.00 
I, 9000001-9000100 9000046. 4 9000050. 5 850.06 30000000.00 
I. 90000001-90000100 90000001 .0 90000050.0 3283.50 4500000000 . 00 


^ The correct result ends in 50.5, and the X in the second formula was calculated in an initial pass through using 
e formula = 2X/N. 
» The correct result is 833.25. 


As can be seen in Table 1, no difference between the formulae for 
the mean occurs until the number of significant digits exceeds six. 
After this point the standard calculator formula yields discrepan- 
cies, whereas the other formula continues to give the correct result 
to eight significant digits. The error occurs much earlier with the 
calculation of the variance. The “definitional” formula is essentially 
correct up to, but not including the eight digit data. The standard 
calculator formula yields incorrect results after the data exceed 
three significant digits! 

The correlation matrices are presented in Table 2 with the results 
of the “definitional” formula in the lower half of the matrix and the 
results of the computational formula in the upper half. Both formu- 
lae give the correct answer when data Set I is correlated with data 
Set II. After that point, the computational formula yields inaccurate 
results. The “definitional” formula is essentially accurate for data 
up to, and including, seven digits. All entries in Table 2 should be 
1.0000, since the last three digits in each data set are ordered from 
001 to 100. Entries which deviate from 1.0000 do so because of the 
effects of truncation error. Table 2 demonstrates, for example, that, 
using a computer program for the ordinary “desk calculator” form- 
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TABLE 2 
Produce Moment Correlations» Using Two Formulae 


Data Set I II Ill IV aA VI 
LEON MG Ma 1.0000 1.0231 5876 +0475 -0049 
II 1.0000 ...... 1.0183 5629 .0205 .0013 5 7 
Ii 1.0000 1.0000 ...... -2966 = —.2097 —.0518 — .0000 
IV 1.0000 1.0000 1.0000 ...... —1.4146 —.2609 —.0000 
Vv 1.0000 1.0000 1.0000 1.0000 ...... —.0309 —.0504 
VI +9901 .9901 .9901 9901 5.0) a 
VII -5038 .5088 5038 5038 -5038 -6202 ee 
Definitional Formula Computational Formula 
(Entries below diagonal) (Entries above diagonal) 
2(X — ŠXY — ¥) N2XY — (2X)\(2Y) 


VAX -XP VY PP Vee (2X)! Vay? — GY 


* The correct result is 1.0000, The X in the definitional formula was calculated in an initial Pi 
through using the formula ¥ = 2X/N. 
b This was a fortuitous accuracy, which resulted from a conjunction of errors, 


ula, the correlation between two identically ordered sets of data one. 
of which consists of three digits (001-100) and the other of five 
digits (90001-90100) is .5876. 


Cautions for Computer Use 
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sponsible for the accuracy of our second formula for calculating a 
mean. It first computed an approximation of the mean, then found 
the average deviation from this value, finally added this to the ap- 
proximation, and thus obtained an accurate solution (even for 
“maximal error” data of eight digits). 

A second approach is to choose statistical formulae very care- 
fully. In addition to not being a research assistant, a computer is not 
just a very fast desk calculator. Formulae most effective for the 
desk calculator are not necessarily the best for the computer. In 
general, formulae should be used which keep the value of a calcula- 
tion, at any one time, as low as possible. For example, formulae 
should be chosen which minimize the use of exponentiation (e.g., 
squaring). 

In conclusion then, truncation error can play an important part in 
calculations. However, if the numbers are small and if the researcher 
is careful, the effects can be minimized. It is clear that computers 
should not be treated as infallible “black boxes,” but should be 
used judiciously, with informed checks of both input and output. 
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Fundamental Statistics in Psychology and Education (4th Edition) 
by J. P. Guilford. New York: McGraw-Hill, 1965. Pp. 605. 
$8.50. 

It is always enlightening to see reflected in a new edition of a 
standard text the effect of a decade’s progress on the author’s in- 
terpretation of his field. To Guilford, the major innovations of the 
last ten years are obviously an increased emphasis on the probabil- 
ity and distribution function bases of statistics and the development 
of inferential techniques of markedly greater power, especially for 
small samples, This view has led to several changes in the content 
and organization of the fourth edition of Guilford’s widely used 
text. 

Most directly, a new chapter devoted to probability and some 
fundamental ideas about distributions precedes the material on 
sampling statistics. The old Chapter 9 has been subdivided into two 
new chapters. The first deals in general terms with estimation and 
inference and the second with significance of differences. In the lat- 
ter, increased and well-deserved emphasis is placed on small-sample 
methods, The treatment of nonparametrics has been expanded to 
include the Kolmogorov-Smirnoy tests, and its emphasis has been 
changed by treating chi-square in a separate chapter. The chapter 
on hypothesis testing gives a new treatment of Type II errors and 
power functions and devotes more space to sample size require- 
ments. For more purely psychological reasons, the chapter on pre- 
diction of attributes has been eliminated and its essentials coupled 
with the accuracy of the prediction chapter. Throughout the book, 
more mathematical demonstrations and proofs have been placed at 
strategic points. A number of these are good, but in general they are 
probably above the heads of mathematically naive students and a 
bit trivial to the sophisticated ones. In the same vein, the repeated 
small inaccuracies in stating statistical concepts will prove irritat- 
ing to more rigorously inclined teachers even though the practice 
makes some topics more comprehensible to the students. 

Guilford has trimmed some excess wood from his first six chap- 
ters and improved them in the process. The detailed descriptions 
of such fundamentals as setting up frequency distributions remain re 
the best source to which introductory students can be sent. The cate 
aloging of facts about measures of central tendency and dispersion 
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is both highly readable and intelligible. It does seem superfluous in 
this day when desk calculators are available to everyone and com- 
puters to many, to teach the trial balance method of calculating the 
mean, standard deviation, and correlation coefficient. It is a com- 
mon view among teachers that the details of these procedures need- 
lessly confuse the main issues. A short discussion of preparing data 
for simple computer programs to perform these tasks would be much 
more appropriate. 

As one would expect, the chapters on correlation and prediction 
are excellent, as is the treatment of reliability. Guilford’s views on 

Validity are, of course, based largely on factor theory, and while the 
concept is thoroughly covered from this angle, a number of other 
ideas tend to be slighted or oversimplified in the process. The final 
chapter on comparability of scores and norms is a sparkling clear 
exposition of a topic that is especially prone to opaque presentation. 

Viewed from the perspective of the total book, one wonders why 
Guilford bothers to include a section on analysis of variance. It is 
clearly a topic of little interest to him and one that is comprehen- 
sively covered in a number of standard sources. This chapter, 
which is uncharacteristically muddy, is marred by a discussion of 
multiple comparisons that includes nothing more recent than 
Tukey’s gap test. This section should be avoided. 

There are a number of other points about the fourth edition that 
deserve mention. Throughout the text, informative references and 
suitable outside readings on various topics are conveniently foot- 
noted. The table of the normal curve is much improved. The prob- 
lem sets are appropriate and useful. The entire book is set in 
unusually clear and readable type and format, and, however irrele- 
Tn to the book as a text, the binding is most colorful and attrac- 

ve, 

Although primarily intended as a text, Guilford’s book will cer- 
tainly serve as a reference volume for many correlation and pre- 


and tetrachoric r, among others. As a text the fourth edition has one 
main drawback. To make a perhaps excusable pun, it is a power 


James A, WatsH 
Iowa State University 
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Statistical Reasoning and Procedures by Robert B. Clarke, Arthur 
P. Coladarci, and John Caffrey. Columbus, Ohio: Charles E. 
Merrill, Inc., 1965. Pp. iv + 390. $7.95. 

The authors of this recent Merrill publication perceive three 
levels of statistical skill: (a) the level at which the student can per- 
form the clerical-computational operations needed in data sum- 
mary and analysis, (b) the level at which there is sufficient under- 
standing of statistical reasoning for the individual to be able to 
interpret statistical results as reported in the literature and to 
carry out research with the aid of a statistician, and (c) the level of n 


specialization. The book has been prepared to help the student ~ * 


acquire a degree of sophistication at the second level, i.e., where he 
can communicate with the specialist and where he can know when 
he needs specialist assistance. > 

The content of the book and the sequence and format of presenta- 
tion are staidly “traditional.” An appeal has been to the non- 
mathematically inclined student so that the approach is primarily 
verbal. Mathematical notation and symbols have not been avoided, 
but they have been kept to a minimum. An appeal has also been 
made, however, to the instructor who adheres to an attempt to im- 
part a thorough understanding of the “bread-and-butter” statistical 
techniques in contrast to the instructor who—in an introductory 
course—attempts to impart a feeling for the limits and the poten- 
tials of statistics. For a first course in statistics, there is more than 
adequate detail: indeed, there is sufficient detail to make this vol- 
ume quite a useful reference at an elementary level. 

The worthwhileness of the purpose of the text cannot be ques- 
tioned. The “middle” level of statistical sophictication is needed in- 
creasingly amongst those entering the major professions in modern 
society. There is no scarcity of texts covering the content of the 
present book, however, so that it does seem reasonable to question 
the need for yet another in the area. The authors make no claim for 
having contributed anything unique or new. The successful payoff 
for their efforts, therefore, seems to depend in great measure on 
whether their particular style and presentation will prove more at- 
tractive than competitive texts. 4 

Happily for the prospects for the book, the author’s prose is 
highly readable. Explanations are patient without being tedious. 
The desire of every author for clarity has been achieved par excel- 
lence in this instance. Numerous diagrams—each worth at least a 
thousand words—have contributed considerably towards under- 
standing, i 

The authors have not been helped in their presentation by the 
printed format. For example, the numerous worked examples (all 
from psychology or education) are carried through in cursive style 
in the printing, rather than placing one step to a line, which would 
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have helped comprehension. Admittedly, this may have been done 
to save on costs, but it detracts from the utility of the exposition. 
Similarly, nothing has been done to draw attention to summarizing 


Little need be said to the initiated user of statistics texts about the 
content of the book, presumably, once the label “traditional” hag 
been affixed. There is the “usual” procession from measures of cen- 
__ tral tendency, through variability, the normal curve, correlation 
and regression, probability, hypothesis testing, ¢, statistical infer- 
ence, and chi-square, The book stops short of the F-test, analysis of 
variance, and non-parametric techniques, 

There are rare occasions for impatience. The first chapter, which 
includes a brief introduction and an even briefer summary of arith- 
metic operations and algebraic conventions, is a poor indication of 
what is to follow. To try to give a reader “some: grasp of certain 
arithmetic and algebraic conventions” in five pages is a forlorn 
hope if he has had no prior acquaintance with these rules and re- 
dundant if he has, Again, the “interested student” is told from time 
to time in the text that he can “consult one of the many available 
treatments” of a given topic—yet there is no bibliography, no sug- 
gested additional or supplemental readings, no references. Each 
chapter ends with a number of excellent problems, but no answers 
are provided, so that there is no chance for the student to obtain 
immediate feedback as to his Success or otherwise, without supple- 
mental effort on the part of the instructor, 

On the whole, however, the organization and substance of the 
content bear little criticism, Perhaps more use could have been 


the text. Any such criticism or minor irritation is heavily out- 
as by the clarity, precision, and thoroughness of the develop- 


This, then, is a book that will not command attention for inno- 
vation or uniqueness. It may find a market amongst present users of 
traditionally-oriented” texts who have become disenchanted, or 


sideration of Persons interested in a well-written, careful exposition 
of the basic statistical techniques. It should also find a useful place 
as a modern reference work in the collection of any instructor of a 
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Statistics in Education and Psychology by Merle W. Tate. New 
York: The Macmillan Company, 1965. Pp. ix + 355. 

Occasionally a book is published for the introductory college 
course in educational/psychological statistics which is basic with- 
out being fatuous. Such a book does not assume mathematical train- 
ing beyond elementary algebra, but it does assume intelligence and 
perseverance. Its author recognizes that although the amount of 
mathematical training in high school and early college has increased 
somewhat in recent years, many of the students who enroll in re- 
quired elementary statistics courses seem to defy this trend. They 


may have been attracted to major in one of the social sciences be- < s 


cause they are more verbal than quantitative in ability and incor- 
rectly perceive social science as non-quantitative. But there is a 
correlation between mathematical sophistication and logical rea- 
soning ability, and many have observed that those students who 
have problems with the algebra in an elementary statistics course 
also have difficulty understanding the non-algebraic concepts. 

Thus, the book which is the subject of this review is elementary in 
that it assumes little mathematical background, but it will not be 
elementary to the mathemaphobe or to one who has an antipathy 
toward a succinct, logical presentation of essentially mathematical 
material. The book is a condensed revision of Professor Tate’s 1955 
Statistics in Education. It gives the deceptive appearance of being 
a small book (355 pages), but within its cover is a thorough, 
tightly-packed presentation of statistics from frequency distribu- 
tion through ¢ test. The F test is not treated, although its chi- 
square counterpart is. 

The author, Merle W. Tate, Professor in the Graduate School of 
Education at the University of Pennsylvania, is obviously a careful, 
accomplished writer and a serious scholar of statistics. The reviewer 
is quite pleased with this book and enthusiastically recommends it 
for courses in elementary statistics. It is not just another statistics 
book but an excellent presentation of much material. The instructor 
will have to work, however, it he plans to use this book, since the 
author wastes few words, and reiteration, exemplification, and 
evaluation by the instructor will be necessary. 

The format of the book is a preface, twelve chapters, references, 
an appendix, a list of 10 tables, answers to selected exercises, and 
an index. The chapters are of unequal lengths, chapters five and 
eleven being relatively shorter and chapters seven, eight, and ten 
being significantly longer than average. The disproportionate 
amount of space devoted to chapters seven and eight reflects the 
rather unusual elaboration, for a beginning statistics book, of ele- 
mentary test theory. Chapter seven might well have been broken 
into two chapters. pie 

There are some differences between the organization, language, 
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and formulas of this text and what one is accustomed to in ele- 
mentary statistics books, For example, a separate chapter is de- 
voted to score transformations, whereas this materia] is usually 


variability, and the use of the term “statistical Series” as a central 
concept. Finally, setting s = V2z?/N instead of s — V32"/(N=1) 
in Chapter IV makes for a certain conflict with traditional presenta- 
tion when the standard error of the mean is defined as Su = s/\/N—] 


The book contains numerous formulas, definitions, and procedure 
descriptions, and it should be useful as a Teference book as well as a 


zation and Presentation of Statistical Data ; II. Characteristics of 
Statistical Series, Central Tendency; IV, Characteristics of Statis- 
tical Series, Variability; V, Transformation of Scores; VI. The 


bution; XIT. The X? Sampling Distribution. 

Chapter I is a good introductory chapter on what statisties is 
and how it should be applied. It discusses the meanings of statistics, 
uses of statistics in data reduction and inference, and sampling. 


the form of a frequency distribution and constructing histograms 
and polygons, Brief descriptions of skewness, kurtosis, and the 
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categories. Chapter VII is an excellent, if lengthy, chapter on corre- 
lation and prediction, although it will require a great deal of 
thoughtful study by the unsophisticated. The product-moment, 
rank order, biserial, fourfold, partial, and multiple correlations are 
all considered in this chapter. Chapter VIII contains much useful 
information on the theory and treatment of test scores. This chap- 
ter will also require careful study. Chapter IX is a brief discussion 
of statistical inference and involves more words and fewer statistical 
symbols than the majority of the chapters in the book. Chapter X 
is a comprehensive chapter on elementary tests of significance em- 
ploying the normal curve tables. Noteworthy in this chapter is 
Table 10.3, which lists the standard errors of 11 different statistics. 
Incidentally, the square root in the denominator of the first formula 
on page 250 should be N, not N—1. Chapter XI is a brief, but 
thorough, chapter on ¢ tests—small sample tests of significance 
of the difference between means and between correlation coefficients. 
The final chapter discusses various uses of the x? distribution, al- 
though the author appears to prefer g statistics to the x? goodness 
of fit for testing the assumpion of normality in a frequency distribu- 


tion. 
Lewis R. AIKEN, JR. 


Trinity University 


Elementary Applied Statistics by Linton C. Freeman. New York: 
John Wiley and Sons, 1965. Pp. viii + 298. $6.95. 

The number of new elementary statistics texts appearing on the 
market is quite astounding. Equally astounding is the variety and 
individuality of presentation of the same basic material. What is 
perhaps a little encouraging in this otherwise overwhelming situa- 
tion is that each new book seems to have optimal utility for some 
particular class of student, so that somewhere amongst the welter of 
new textbooks each student should be able to find a book very much 
suited to his own individual needs. 

Freeman’s new book appears to be oriented towards the student 
who is primarily interested in applying the basic statistical tech- 
niques, in having some understanding of the reasoning behind sta- 
tistical usage, rather than an appreciation of the theoretical devel- 
opment of the statistics, As the author is quick to point out, it is 
neither a reference work nor a “technical essay” in statistics. To 
categorize the text as a “cookbook” would be unkind; yet in general 
strategy, its development occasionally comes close to being just 
that. 

Since the author chooses to define science as the “study of rela- 
tionships among variables,” the text on the one hand places a heavy 
emphasis on associational techniques. Some very interesting juxta- 
positions and relationships result from this approach, with a high 
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potential for student insight. On the other hand, there id a strong 
emphasis on levels of measurement (following Steven’s categori- 
zation), and most of the presentation is organized around relation- 
ships within and between the various levels. The two-track devel- 
opmental sequence—the nature of the problem and the form of the 
data—has been employed to produce a readable, often’ attractive, 
text. 

The book is divided into four sections a section concerned with 
the background, or “entry behavior” needed for the study of statis- 
ties; a section about summarizing distributions on a single vari- 
able; one on describing the association between two variables; and 
a final section on statistical inference, d 

The first section—the background section—consists of three chap- 
ters which attempt to cover a great deal of ground in a few pages. 

„Chapter one is devoted to the definition and illustration of various 
levels of measurement, despite its rather misleading title, “The 
Subject Matter of Statisties.” Interestingly, only nominal, ordinal; 
and interval scales, are considered. The distinctions between the 
scales are concisely and clearly presented, each scale being intro- 
duced by a brief definition, with further exemplification. Unfor- 
tunately, the positioning of the chapter in the general developmen- 
tal sequence leaves the concept.of scales of measurement somewhat 
dangling, and for the moment unrelated to the primary purpose of 
the book. Chapter two indicates very briefly some of the major uses 
(and misuses) of statistics, and distinguishes between descriptive 
and inferential statistics. The final chapter in the introductory sec- 
tion contains the content that so often seems necessary for an ele- 
mentary course; yet, to anyone with any sophistication in mathe- 
matics, it seems too hopelessly sketchy to be of much use. In some 
eleven pages, chapter three attempts to outline “the basic arithme- 
tic and symbolic skills that are required.” There is about a page on 
summation notation and another two pages on the “rules” for the 
use of sigma; arithmetic is “reviewed” in two and a half pages; 
and the remaining four or five pages are devoted to parenthetical 
gems on how to read the Square root table in the back of the book, 
how to round numbers, and how to compute proportions and per- 
centages. 

Granted that a large Proportion of students taking elementary 

atisties in the social sciences are not numerically adept, it is ex- 


after all, a branch of applied mathematics. As such, it is of doubtful = 
value to try to disguise this fact and to bend over backwards in ~ 


- = 
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order to avoid the use of anything that might be considered “math- 
ematics.” Chapter three in the present book does little to reassure 
or to enlighten the unsophisticated. It may better have been either 
omitted or considerably expanded. 

The second section of the book is concerned with summarizing 
distributions on a single variable. The section begins with an over- 
view which introduces a number of key terms and briefly reviews 
graphical techniques in aynon-quantitative fashion. The continued 
protection of the reader from the bogey of actual numbers seems 
unnecessary: for example, it prevents the author from devoting 
more than eleven lines to frequency polygons and histograms. The 
reader is also saved the mental effort of struggling with such an 
esoteric term as “kurtosis.” “Peakedness” is regarded as sufficiently 
taxing. 

Having survived thus far, however, the reader should find re- 
maining chapters in the book to be considerably more inspiring. 
Continuing with the second section, chapter four discusses methods 
of summarizing distributions on nominal scales. The mode and 
variation ratios are defined and illustrated with excellent examples, 
and useful notes are provided indicating alternative equivalent 
statistics (for example, the “uncertainty measure” and Mueller and 
Schuessler’s Index of Qualitative Variation). Chapter five presents 
definitions and illustrations for computing the median and decile 
range for ungrouped data while chapter six does the same for the 
mean and standard deviation. Although there is only one illustra- 
tive example for each technique, it has been well chosen and well 
expounded. In each case, the raw data for the example have been 
culled from the literature, which adds to the readability. Again, and 
. throughout the book, reference to alternative statistics is most 
valuable. ; x 

Section C in the text is concerned with describing the association 
between two variables. The section goes through the six combina- 
tions of the three levels of measurement two at a time—sometimes 
a little glibly, but more often with considerable illumination. There 
is always something exciting about finding fresh material in an 
elementary text; and in this third section, the book provides a 
number of unusual encounters. For example, in considering the as- 
sociation between nominal scales, we meet Guttman’s Coefficient of 
Predictability and mention of Tschuprow’s T—surely an interesting 
innovation. Again, at the level of ordinal measurement, Goodman 
and Kruskal’s Coefficient is presented in detail, while tau and rho 
are dispensed with in a total of eight lines—and both without for- 
mulation or exemplification. Person’s r receives about a dozen pages’ 
attention and is treated clearly and fully. For describing the asso- 
ciation between a nominal and an ordinal scale, a modification of 
the Wilcoxon test is proposed (the author’s own extention) ; for the. 
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association between a nominal and interval scale the correlation 
ratio is recommended; while passing mention of the biserial co- 
efficients and “point-multiserial correlation” round out an extensive 
exposition of Jaspen’s coefficient of multiserial correlation” as the 
appropriate index for describing the association between an ordinal 
and an interval scale. The author’s choice of indices has in each 
case been fully and clearly presented, and an example worked 
through. One might query the wisdom of the choice—or perhaps, 
better, the wisdom of the omission—in a few instances, but one 
of the major attractions of the book lies in its proposed use of 
some of these lesser-known indices. It is here that the avoidance 
of mathematical derivation may, however, be of maximum disad- 
vantage. Without reference to the original source (and in most cases, 
none is given in the text) it may be difficult for a student to gen- 
eralize beyond the specific example given. Without in any way try- 
ing to detract from a refreshingly innovatory approach, there is 
here a slight suggestion of that “cookbook” strategy mentioned 
earlier, 

The fourth section of the book is an attempt to outline the rudi- 
ments of statistical inference. The section begins with a chapter 
which is concerned primarily with defining terms and illustrating 
the processes involved in statistical inference. The discussion, which 
is brisk and readable, follows a traditional pattern. The thirteenth 
chapter deals with testing hypotheses about two ordinal scales—in 
effect, testing the significance of the Goodman-Kruskal coefficient. 
Rho and tau receive only five lines this time. Testing hypotheses 
about two interval scales is similarly almost exclusively concerned 
with testing for the significance of r. Chapters 15-17 are concerned 
with the testing of hypotheses of the combination of scales: nom- 
inal-ordinal, nominal-interval, and ordinal-interval, The U-test 
and F come in for major attention, with very brief references being 
made to the runs test, median test, and—oddly—the t-test. Chapter 
eighteen returns to the problem of testing hypotheses about two 
nominal scales and discusses chi square in some detail. The presen- 
tation is consistently lucid, and one must comment again on the 
excellence of the illustrative examples. Some of the emphases, to the 
total exclusion of more traditional techniques, might leave a number 
of gaps for the student who is confronted with the task of reading 
and interpreting research literature. While bearing the deficiencies 
in mind, what is included in the text has been well done. 

The text comes to a rather sudden end after the fourth section. 
There is a brief bibliography, primarily of an eclectic nature. Cer- 
tainly the student who wished to pursue a specific topic would get 
little guidance from the list provided. More specific references at 
the end of each chapter would have been beneficial. A list of sta- 
tistical symbols and the statistical tables referred to in the text, 
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together with a selection of data tables for use in problems, com- 
plete the appendix. A few problems (never more than ten) asso- 
ciated with each chapter and the answers to the odd-numbered 
problems also appear at the end of the book. The problems are 
simple, requiring minimal arithmetical manipulation, and are spe- 
cifically related to the textual material. A brief index completes the 
book. 

In making an over-all evaluation of the book, a return must be 
made to the title, Elementary Applied Statistics. It is difficult to 
decide at what point the book would find most use. It appears to be 
too elementary for a graduate course, yet some of the techniques it 
provides are unlikely to be utilized at an early stage in a behavioral 
science program. In many ways, the book appears to be a useful 
supplement to a more traditionally-oriented text. It makes a unique 
contribution to the flood of elementary texts in introducing a num- 
ber of less well-known but powerful tests. Its style is highly read- 
able and the illustrations are excellent. Therefore, it should be 
capable of rewarding individual study. And it is in this sense that 
the book is seen as being potentially valuable as supplementary 
reading. Its over-avoidance of mathematics, the lack of adequate 
references, and the relatively simple problems reduce its appropri- 
ateness as a primary text. Superficially, it would also appear to 
restrict teaching very much to the direct content—for those who 
dislike this strategy, some discomfort may be engendered. Again, 
this could be avoided were the book to be used in a supplementary 
role. 

Elementary Applied Statistics must be commended for its fresh 
approach and for its expository style. It would deserve the attention 
of instructors of elementary statistics courses as valuable supple- 
mentary reading. } j 

PETER A. TAYLOR 
University of Illinois 


Foundations of Behavioral Research: Educational and Psycholog- 
ical Inquiry by Fred N. Kerlinger. New York: Holt, Rinehart 
and Winston, Inc., 1964. Pp. xix + 739. 

This compendious volume on behavioral science methodology 
should foster the growing sophistication and efficiency of students 
looking forward to conducting psychological, sociological, and edu- 
cational research. It may be compared favorably with such previous 
methodological books as Underwood’s Psychological Research and 
Festinger and Katz’s Research Methods in the Behavioral Sciences. 
It is a far more comprehensive and difficult treatment of the ma- 
terial than the recent introductory text by Scott and Wertheimer. 
This book is suitable as a text or as a reference for upper division 
college and graduate students; it would be adaptable for a two 
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semester course on methodology when supplemented with additional 
„references; it would provide valuable supplementary reading for 
_ courses on statistics and experimental psychology. 

" Kerlinger states that the major purpose for the book is to help 
students understand the fundamentals of the scientific approach 
to problem solving. The author outlines the scope, objectives, and 
subject matter as follows: (1) It is a treatise on scientific research. 
(2) It concentrates on helping the student to grasp the difficult re- 
lations between a research problem and “the design and metho- 
dology of its solution.” It is not a methods book, but refers the stu- 
dent to references on the specific methods considered. Throughout 
the book there are detailed discussions on (a) stating research prob- 
lems, especially on developing testable hypotheses; (b) selecting an 
appropriate and powerful design for planning for observations and 
analyzing the data; and (c) pointing out the logical relations be- 
tween the problem under investigation and the research method 
used. (3) The content and the explication of research procedures 
are linked together through the notions of set, relation, and vari- 
ance, These ideas, combined with the fundamentals of set theory, 
probability theory, statistical design and analysis, and measure- 
ment, are used to integrate the diverse content of research activity 
into a systematic and meaningful whole. (4) The book concentrates 
on educational and psychological problems, especially the psycho- 
logical aspects of educational research problems. 

The appropriateness of the methodology to the problem under 
investigation is based upon careful consideration of (a) a choice of 
research design, (b) methods of observation, (c) methods of meas- 
urement, and (d) types of analysis from which interpretations are 
derived. In other words, the design, methods of observation and 
measurement, and the statistical analysis must be appropriate to 
the research problem and should be chosen and combined in a way 
so as to obtain the maximum amount of information from the data. 
Significant research stems from the complex chain of theory, de- 
ductions from theory in the form of hypotheses, design, measure- 
ment, and analysis. The book is divided into eight main parts, each 
one ening an important aspect of the logic and procedures of re- 
search. 

Part One discusses the salient characteristics of the scientific ap- 
proach, “Scientific research is systematic, controlled, empirical and 
critical investigation of hypothetical propositions about the pre- 
sumed relations among natural phenomena.” Definitions and ex- 
amples of different types of hypotheses, constructs, and variables 
are presented in detail. The notion of randomness and sampling 
from sets is included in the introductory section. 

Part Two briefly treats the main principles and concepts of set 
theory as a way of giving the student an appreciation of the formal 


BOOK REVIEWS 219 


language underlying the important operations ‘of scientific metho- 
dology. The consideration of variation as a universal character- 
istic of natural phenomena leads to the definitions of ‘different kinds 
of variance, including population and sample variances, systematic 
variance, experimental or between-groups variance, and error vari- 
ance, Examples of systematic and error variance lead intuitively 
to the notion of statistically significant differences. The operations 
that can be performed with variances are summarized under the 
categories of components of variance, covariance, and common- 
factor variance. 

Part Three, comprising approximately one fifth of the book, sum- 
marizes the basic concepts of probability theory and statistical 
inference, emphasizing the role of statistical hypothesis testing and 
analysis of variance in behavioral science research. The procedures 
outlined in this part give primary consideration to the rationale for 
determining significant statistical differences, although the author 
stresses in a single sentence the need for consideration of estima- 
tion, interval estimation, confidence intervals, and exact probability 
methods, since they give the research worker more information 
about the relations found in the data. Several paragraphs are de- 
voted to an interesting discussion of the relation of substantive 
and null hypotheses. The author gives somewhat limited attention 
to the meaning and importance of interaction in analysis of vari- 
ance designs. 

Part Four, Designs of Research, offers a set of principles, which 
every student of statistical and research design would benefit from 
reading. Research design is seen as having two basic purposes: (a) 
to provide answers to research questions, and (b) to control vari- 
ance. Research designs, generally, should maximize experimental 
variance and minimize error variance. Kerlinger discusses ‘six steps 
by which these two goals may be reached. Chapter Sixteen, “Poor 
Designs,” is one of the highlights of the book, in the reviewer's 
opinion. 

Part Five contains an extensive discussion of the characteristics, 
criteria, advantages and disadvantages of various types of research, 
ranging from laboratory experiments, field experiments, field studies, 
and surveys. He differentiates between experimental research and 
ex post facto research on the basis of the kinds of controls, sam- 
pling, and interpretations of data characteristic of each. He points 
out that behavioral research frequently must be content with using 
ex post facto methods, but the precision and scientific validity in 
the use of such methods may be increased if research workers would 
concentrate upon a set of alternative hypotheses and then treat the 
results and the interpretations made therefrom cautiously. 

Part Six deals with the foundations of measurement, including 
levels of measurement and scales. Then reliability theory is intro- 
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duced as an extension of the notions of set, relations, and variance. 

Reliability is defined as “the proportion of the true variance to the 

` total obtained variance of the data yielded by the measuring in- 

` strument.” Conversely, “reliability is the proportion of error vari- 
ance to the total obtained variance of the data yielded by a meas- 
uring instrument subtracted from 1.00, the index 1.00 indicating 
perfect reliability.” The statement that reliability is a necessary 
but not sufficient condition of the value of research results, leads to 
the consideration of types of validity. Validity is defined as the pro- 
portion of the total variance of a measure that is a common factor 
variance, Kerlinger summarizes this part of the book as follows: 
“Poor measurement can invalidate any scientific investigation. 
Most of the criticisms of psychological and educational measure- 
ment... center on validity. . . . Achieving reliability is to a large 
extent a technical matter. Validity, however, is much more than 
technique. It bores into the essence of science itself. It also bores into 
philosophy. Construct validity, particularly, since it is concerned 
with the nature of ‘reality’ and the nature of the properties being 
measured, is heavily philosophical.” 

Part Seven, Methods of Observation and Data Collection, classi- 
fies and discusses a heterogeneous assortment of methods by the 
degree of their directness. Among the methods reviewed are inter- 
views and questionnaires, objective tests and scales, critical inci- 
dent technique, the problems involved in defining units of behavior, 
and the scales appropriate to various units, rating scales, projective 
methods, and the use of available relevant materials and the prob- 
lems involved in content analysis, sociometric methods, semantic 
differential, and Q Methodology, 

Part Eight briefly describes the processes of inference involved in 
data analysis and interpretation, and then presents still further 
procedural methodology, including relevant methods from set the- 
ory and statistics. A short section on the interpretation of research 
data succintly summarizes many of the research precepts which 
Kerlinger wishes students to grasp. This part of the book includes 
chapters on the analysis of crossbreaks (introduced in the first part 
of the book) and factor analysis. The use of factor analysis to test 
hypotheses about the relations among variables in addition to that 
of exploring and identifying variables is described. 

The book concludes with three short appendices, the first one out- 
lining some main points on report writing and pertinent references, 
the second one reviewing the differences between so called historical 
and methodological research, and the last one very briefly indicat- 
ing the value of electronic digital computers in conducting behay- 
ioral research. 

The book contains both subject and author indices, which are 
fairly accurate. Several typographical errors were evident. In sev- 
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eral contexts the reviewer would have welcomed a more extensive 
and varied use of illustrative research reports. This book, to which 
the author has given a great deal of thought, deserves a place in 
collegiate and students’ libraries. 

CHERRY ANN CLARK 

The Meyers Clinic, Los Angeles 


Psychological Tests and Personnel Decisions (Second Edition) by 
Lee J. Cronbach and Goldine C. Gleser. Urbana, Illinois: Uni- 
versity of Illinois Press, 1965. Pp. viii -+ 347. $7.95. 

The potential impact of decision theory on testing theory and 
practice is great, and in their 1957 edition the authors spelled 
out many of the important changes in viewpoint and practice likely 
to take place when testing is regarded as a part of a decision-mak- 
ing process and not as an end in itself. A check of some past issues 
of EDUCATIONAL AND PsycHoLocicaL MuasurEMENT indicates that 
this new revised edition of this still basic volume may be urgently 
needed for another reason in addition to that of up-dating: few 
references to the original edition were found, although references to 
Cronbach’s Essentials of Psychological Testing were frequent. With 
a book as important as this one it would be very desirable if the 
ratios of mention were reversed, with familiarity with “EPT” taken 
for granted, but “PTPD” cited frequently as a stimulus for work 
in the broader context of decision theory. 

In this Second Edition, the authors have added a valuable thirty- 
page summary of recent work in decision theory related to selection 
problems and to the use of tests. They reported that “while the 
ingredients represented in our survey might conceivably be com- 
bined into some great new insight, we cannot yet report that we or 
others have made any such advance over our first edition position. 
Rather, we see recent writings as pushing ahead toward more 
definite applications of certain principles in actual situations, and 
in another vein, toward extremely abstract analyses of various 
problems that, move farther and farther from application.” 

Included in a supplement are representative and valuable papers 
by other authors, some new, some reprinted, illustrating possible 
applications of decision theory to selection problems as well as 
raising (inevitably) the methodological and practical issues in- 
volved. Although much of the treatment is mathematical, the gen- 
eralist in measurement is given an instructive view into the boiling 
cauldron of flickering assumptions and mathematics too hot to 
handle except for those with much training: he is likely to be well 
convinced of the potency of the decision theory approach, and grate- 
ful that the fires of criticism will sear away the dross. However, he 
may have his present commitment to classical concepts in testing 
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(as a matter of sound, more sound, and most sound measuring in- 
struments) somewhat scalded. The wound will heal. For quite some 
time, the new tissue may not be so strong as it was before, but it 
will be more sensitive to the realities of the world in which the tester 
lives, Greater perception of how testing fits into the real world will 
come, it seems, when one’s toughened measurement, callouses are 
replaced by receptors more sensitive to the storms and hail of a 
difficult world where tests are used to make decisions in the dim 
light of uncertain utilities. 

One should be sensitive, the authors urge, to the fact that a test 
with high validity (such as an aptitude test predicting college 
grades) may not be so useful as it seems, since the test is competing 
with high school grades as a predictor when this (free) information 
overlaps considerably with test score predictions. The authors do 
not necessarily mean that such aptitude tests are of diminished 
value; they do indicate that “The characteristics of the specific de- 
cision determine ‘what the test is worth’,” and that “a test of valid- 
ity .20 in one situation may be more beneficial than a test of valid- 
ity .60 in another.” 

Decision theory leads them to important discussions of alterna- 
tive strategies in single-stage, two-stage and multiple-stage decision 
making (e.g., in selection problems) and to point up the “‘band- 
width-fidelity” dilemma. It is suggested that for some purposes a 
broad band technique such as an interview, touching on many as- 
pects of the interviewee, may provide more over-all utility even 
though each of the many items of information it provides is less 
reliable than a single score from a very reliable ability test. The 
authors are of course not arguing against sound measurement prac- 
tices. They are arguing that measurement is for a purpose the utili- 
ties of which should determine what optimum degree of accuracy 
in measurement is most useful. 

The authors also point out the importance of adaptive treatment 
as an alternative to better selection. “The job simplification expert 
and the human engineer seek to fit the job to unselected men. The 
greater their success the less the value of selection. The tester has 
failed to realize that he is competing with the treatment simplifier.” 

Emphases such as those abstracted above have of course been 
explicitly and implicitly foreshadowed in the literature for years, 
and the authors do cite such previous work. But for the person in- 
terested in testing, in personnel problems, and in learning at this 
time something of what graduate students in testing will probably 
be routinely required to learn in 1970, this book is a must. For those 
who have already acquired the 1957 edition, the supplements and 
the guide to the relevant literature from 1955 to 1963 should make 
the book worth the re-investment. As an up-to-date discussion of 
the anticipated huge impact of decision theory on personnel prob- 
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lems’ in general and testing practice in particular, this book is 
strongly recommended. 
T. B. SPRECHER 
Educational Testing Service 


Studies in Item Analysis and Prediction by Herbert Solomon (Edi- 
tor). Stanford, California: Stanford University Press, 1961. 
Pp. xi + 310. $8.75. 

Constituting an important addition to the Stanford Mathemati- 
cal Studies in the Social Sciences Studies in Item Analysis and 
Prediction is a collection of 19 papers by ten renown contributors 
to the professional literature in mathematical statistics. Edited by 
Herbert Solomon who for many years has been quite interested in 
the application of statistical methodology to educational and psy- 
chological measurement, this volume is primarily concerned with 
the development of mathematical models that describe optimal de- 
signs and approaches to problems of prediction and classification. 
In particular emphasis is placed upon the formulation of optimal 
test designs in terms of which items with what are deemed optimal 
properties may be selected, although the computational difficulties 
associated with use of the models have placed serious limitations 
upon the practical length that a given test can assume. Nevertheless, 
one may anticipate that progress both in mathematical decision 
theory and in computer technology may eventually make feasible 
the application of several of the models proposed. Certainly this 
volume lays an impressive and basic foundation on which addi- 
tional creative research efforts in test theory and in test analysis 
can be expanded. 

Preceding the three major divisions of the volume is an introduc- 
tion by Herbert Solomon—an overview of the entire book. In a 
highly lucid style Solomon not only has outlined a rationale under- 
lying test theory, but also has provided a succinct statement of the 
scope and significance of the contributions of each chapter often in 
its relation to the contents of other chapters of the book. In fact, his 
discussion of each paper, which allows the psychologist with limited 
mathematical experience to gain the gist of each contribution, serves 
to clarify and to amplify in a meaningful way what has sometimes 
been only implied or somewhat obscurely developed in several of the 
papers. 

In Part I the first seven chapters are concerned with item selec- 
tion procedures within the framework of a multivariate normal 
structure. The first three chapters form a logical unit in which prob- 
lems of reliability and validity (including the attenuation para- 
dox) are considered. In terms of the assumptions of a unidimen- 
sional trait, dichotomous scoring of items, the description of item 
characteristics as normal ogives, and a normal distribution of the 
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measured trait in the examinee group, a theory of test design is 
systematically developed. The remaining chapters in Part I are 
largely concerned with the problem of optimal selection of test 
items without the need of determining the almost infinite number 
of possible multiple correlation coefficients associated with almost 
countless composites of test items. i 

The development of nonparametric probabilistic models for 
handling dichotomous systems of data and for classifying students, 
especially in relation to modern statistical decision theory, furn- 
ishes the main emphasis underlying Chapters 8 through 14 in Part 
II. In Part III the inherently difficult W classification statistic de- 
vised by T. W. Anderson and the determination of its approximate 
sampling distribution are treated at length in the last five chap- 
ters. The mathematical complexities in the chapters of Part III are 
formidable almost beyond belief. 

What the impact of this definitive volume will be upon future de- 
velopments of test theory remains to be seen. Until greater numbers 
of psychologists receive the requisite training in higher mathema- 
tics which one would estimate to be at a level considerably in excess 
of an M. A. degree in mathematics additional theoretical develop- 
ments will be somewhat limited. Perhaps more serious than the 
absence of psychologists with mathematical training approximat- 
ing the doctoral level is the lack of a necessary group of individuals 
with intermediate levels of mathematical maturity and with sub- 
stantial interests in measurement and evaluation who can translate 
these esoteric theoretical developments into the day-to-day programs 
of personnel selection and classification. Unfortunately, it is not un- 
likely that at least twenty or thirty years will expire before any 
substantial or noteworthy proportion of the theoretical develop- 
ments of this truly significant book will become operational in local, 
state, or national testing programs. 

Witu1am B. MICHAEL 
University of California, Santa Barbara 


Spatial Ability: Its Educational and Social Significance by I. Mac- 
farlane Smith. San Diego: Robert R. Knapp, 1964. Pp. 408. 

American education appears to be faced by some of the same 
problems that are faced by education in Great Britain. This book 
proposes an approach to one of these problems that should be of in- 
terest to educators and psychologists in both countries. The book 
begins with a discussion of the shortage of scientific manpower and 
then proposes that this could be alleviated if technical schools 
would give consideration to spatial ability in their selection of stu- 
dents. The disregard of spatial ability by schools results not only 
in a shortage of trained technical manpower, but also in the pres- 
ence in the schools of a group of children who possess talents they 
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are not called upon to use. The rest of the first half of the book is 


concerned with factor-analytic studies of spatial ability. In the 
second half of the book there are discussions of the relationship of 
spatial ability and abstract ‘thinking, temperament, motor per- 
severation, critical fusion frequency, alpha rhythm, and the reticu- 
lar brain stem formation. The appendix includes.a number of short 
biographical sketches of famous persons whom Macfarlane Smith 
has found to illustrate his conception of spatial ability or the lack 
of it. 

According to Macfarlane Smith, spatial ability is a talent of 
crucial importance to the prospective scientist or technician. It is a 
talent which is neither recognized nor exercised appropriately in 
school. This reviewer judges that he had a good case. The book, 
however, is more hortatory than dialectic—the author cites study 
after study and case after case in a way that seems more likely to 
importune than to clarify the reader’s mind. Since this book is ob- 
viously addressed to educators as a primary audience, this approach 
may be an appropriate strategy. However, to the scientist this style 
may be somewhat less appropriate. Smith has accumulated a great 
deal of information pertinent to his topic from a wide variety of 
studies. A more efficient organization of all of this material would 
have presented problems. 

Having had some experience and previous interest in this field, 
this reviewer read the book primarily in the hope of at last learning 
what “spatial ability” was. In this respect the book was, perhaps 
inevitably, a disappointment. Smith does not make this a central 
issue. On page 55 we find this statement: “... the special aptitude 
(spatial ability) . . .would be manifested in an ability to perceive 
and reproduce shapes correctlty, i.e. with their dimensions and their 
relations in due proportion.” This is about as complete a definition 
of the trait as is offered in the book at any single point. However, at 
various places there are additional qualifying remarks, such as: 
“spatial ability is quite as intellectual as verbal or numerical 
abilities” (page 36); or “. . . there is a correspondence between... 
spatial ability . . . and a ‘fixative’ or ‘concentrative’ mode of atten- 
tion . . . a tendency to attend to a configuration as a whole . . . mas- 
culine attitudes . . . schizothmia . . . emotional stability” (Page 238). 
Although this discussion may represent a marked advance over 
previous definitions of spatial ability, it appears that there is still 
much to be learned. 

This book is a clear demonstration that factor analysis is a very 
permissive technique—not only in the type and amount of rotations 
and other mathematical operations, but also in the psychological 
interpretation that may be deduced from the numerical tables. The 
psychological interpretations are subjective. Moreover, they are in- 
fluenced both by one’s knowledge of the nature of human nature and 


226 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


by one’s familiarity with tests as well as by one’s attitude toward 
the meaning of tests. Smith’s contribution is that he has brought 
together a very broad knowledge of tests with what appears to this 
reviewer to be a sensible view of the nature of man. Since among 
psychologists there is little consensus regarding the nature of man, 
Smith’s interpretations will not be convincing to all psychologists. 
Fortunately, the data cited by Smith are not restricted either to 
studies of cognitive tests or to the results of factor analyses. He in- 
cludes chapters on temperament and on various physiological studies, 
This breadth of view is one of the main virtues of the book. 

This book may be commended to the attention of persons other 
than just those interested in spatial ability. The problems and 
challenges faced by those interested in spatial ability twenty years 
ago have something in common with the problems and challenges 
faced by the students of “creativity” today. And the case that can 
be made for the importance of spatial ability to education can, per- 
haps, be made for other talents as well. Macfarlane Smith proposes 
that education may be too exclusively verbal. Surely he is right, al- 
though psychologists may have yet to discover how to educate the 
verbally inept. Anyone who wishes to broaden the base of educa- 
tion may find some interest and value in this book. 

Cuartes T. Myers 
Educational Testing Service 
Princeton, New Jersey 


The Structure and Measurement of Physical Fitness by Edwin A. 
Fleishman. Englewood Cliffs, N. J.: Printice-Hall, Inc. 1964. 
Pp. 207. 

Examiner's Manual for the Basic Fitness Tests by Edwin A. 
ae Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1964. 

p. 60. 

Historically, there have been alternating waves of public interest 
in the subject of physical fitness, with concern high during war or 
threat of war, and low during periods of comparative peace. Today 
the attainment of a high level of fitness is an international aim. 
With reference to the current concern, the United States (at the 
White House level) was first shocked into action by a July 1955 re- 
port (widely publicized by the press) which indicated that Eur- 
opean children were more fit than their American counterparts. 
Though attempts to identify levels of fitness are far from new, dat- 
ing back to the latter part of the nineteenth century, the seriousness 
of the contemporary situation has sparked innumerable efforts to 
devise adequate physical fitness tests. 

Dr. Fleishman, currently Director of the Washington Offices of 
the American Institute for Research and a consultant to the Sur- 
geon General of the Army, has recently published the results of a 
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series of studies prepared under contract with the Office of Naval 
Research. The investigations were centered at Yale University, 
where from 1957 to 1963, Fleishman was Director of the Human 
Skills Research Project and Professor of Psychology. The book and 
manual reviewed below contain the “lay” version of the results of 
Fleishmen’s extensive technical investigations. 

Fleishman and his associates attempted, with effectiveness, to 
isolate dimensions of physical proficiency. Starting with factors 
which through the years have been empirically deduced and, in 
some cases, previously identified by physical educators through fac- 
tor analytic means, Fleishman located and/or postulated fourteen 
dimensions of proficiency. Each was then intensively studied by 
means of the statistical technique of factor analysis. 

An attempt was first made to conceptualize a strength and en- 
durance area, The parameters were found to include “dynamic 
strength” (the ability to exert muscular force repeatedly, or for 
long periods of time). This factor was seen to be the most general 
in the strength domain; “trunk strength” was found to be a 
specific secondary dynamic strength component. Also identified 
was “static strength” (commonly now called “isometric strength— 
the ability to exert force against heavy or immovable objects). A 
third parameter was termed “explosive strength” or “energy mobili- 
zation” (probably synonymous with the physical educator’s term 
“power”—the expenditure of force). 

Next were tackled performances which apparently emphasize 
chiefly speed, flexibility, balance, and coordination. Components 
identified were two kinds of flexibility—‘extent flexibility” (the 
ability to stretch, flex, and extend as far as possible) and “dynamic 
flexibility” (flexibility in action); “speed of limb movement;” 
“gross body equilibrium ;” balance of both the static and dynamic 
type which depend chiefly upon nonvisual cues, and balance which 
requires visual cues. In the tasks chosen to represent this speed- 
flexibility-balance-coordination area, explosive strength, previ- 
ously identified, again emerged. Agility, however, was not identified 
as an independent factor. 

Thus Fleishman identified seven factors of physical proficiency: 
extent and dynamic flexibility; explosive, dynamic, and static 
strength (with trunk strength as a secondary component); gross 
body coordination; and gross body equilibrium. It was hypothe- 
sized that cardiovascular-respiratory endurance should, on logical 
grounds, be added to this list. 

In addition to describing the details of the analysis of tests for 
each of these categories, The Structure and Measurement of Physi- 
cal Fitness includes general discussions of ‘Abilities and Motor 
Skill,” “What Do Physical Fitness Tests Measure?”, specific in- 
formation regarding Fleishman’s subsequent work establishing na- 
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tional norms for the tests which he suggests as measures for each of 
the identified factors, developmental curves in relation to age 
and sex, and recommendations for a battery of basic fitness tests, 
The manual presents, in concise form, test descriptions, admini 
trative procedures, evidence of validity and reliability, and national 
norms for boys and girls, age 12-18, based on performance records 
of 20,000 young people in 45 cities across the country. H 
This work is monumental in scope, the most extensive and in- 
tensive effort yet attempted to isolate dimensions of proficiency. It 
confirms factors previously identified by other investigators; in q 
addition, new factors have been isolated and some old ones ree. 
defined. j 
Nowhere is the larger concept of Total Fitntess mentioned (fitness, 
broadly conceived, involves the integrity of the total organismi 
social, mental, emotional, and spiritual aspects as well as physical 
No doubt Fleishman’s work will be criticized because of this lack. 
This reviewer feels, in Fleishman’s defense, that he never intended 
to imply that the physical aspect of fitness constitutes its totality; 
the name of the book properly limits its scope. 
Perhaps an important and ultimate value of Fleishman’s efforts, 
though not his purpose, may be recognition that his data provide ~ 
the most conclusive evidence yet advanced for belief in the extreme © 
specificity of motor performance. 
Because the publication of the results of this voluminous amount ~ 
of investigation occurred only after a nationwide motor fitness — 
testing program had already been not only initiated but also well 
advanced by the American Association for Health, Physical Educa- 
tion, and Recreation, Dr. Fleishman’s recommendations may find 
their way into practice rather slowly. This possible lag is unfortu- 
nate. His battery of tests requires little or no equipment. Moreover, 
the tests can be readily administered and scored by almost any- 
one who gives them careful study. 
ÅILEENE LOCKHART 
University of Southern California 


D. Glock. Boston: Allyn and Bacon, 1963. Pp. 640. 


The volume, Evaluating Pupil Growth, is intended to be used 
as a textbook 


Evaluation for s 
the elementary 


for a one-semester course in Measurement and 
tudents who are preparing to be teachers in either 
1 or the secondary school. Evaluation is broadly 
interpreted to mean the appraisal of the degree to which the ob- 
Jectives of the educational program are reached as they affect 
pupil growth. Evaluation is assumed to include the use of all types 
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Evaluating Pupil Growth by J. Stanley Ahmann and Marvin 
of techniques and instruments, tests and non-tests. i 
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This volume is the second edition of a textbook first published in 
1959 by the same authors. From comment made in the preface one 
can assume that certain professors and students who used the 
first edition of this book made recommendations for improvements 
some of which were implemented in this second edition. 

Evaluating Pupil Growth possesses some characteristics which 
make it unique as a textbook in this field. One of these unique 
characteristics is the arrangement of the chapters. Most textbooks 
in Measurements start with a chapter such as Measurement Today. 
Then follow chapters on the History of Measurement, Statistics, 
Validity and Reliability, Teacher-made Tests and Standardized 
Tests. This book postpones the Validity and Reliability chap- 
ters until after the chapter on the Teacher-made Tests. To be able 
to do this the authors do have to discuss these two topics, although 
briefly, before the chapter on Teacher-made Tests. They then de- 
vote an entire chapter to each of the topics, Validity and Relia- 
bility, in the final half of the book. The reviewer is not commenting 
adversely about this arrangement of major topics of the book; he 
is simply saying that it is unusual. Furthermore, a professor in 
teaching a course may take the chapters in any sequence he wishes. 

Another unique characteristic of Evaluating Pupil Growth is the 
use of questions and exercises called problems placed appropriately 
throughout each chapter. To the reviewer this gives the book a real 
strength. Of course, the problems may be used as the teacher of the 
class dictates, but the student who reads and answers them as 
they apear during the course of reading the chapter will find them 
a distinct help in enhancing his understanding of the content. 

A third unique feature of the book is the use by the authors of 
novel, well selected, interesting, narrative or descriptive, stories or 
discussions at the beginning of each chapter. These are all appro- 
priate to some of the content of the chapter. The first chapter, for 
example, begins with a summer beach scene where the physical and 
psychological differences among the beach population first and then 
among people in general are described. The reading material then 
moves on to the differences among pupils in a school and finally 
to the need to evaluate these differences. Each chapter in a similar 
manner begins with illustrative material to stimulate interest and 
then leads into the topic of the chapter. As one reads the book, he is 
delighted with these little digressions and often thinks forward 
in anticipation of what anecdote or illustration will introduce him 
to the next topic. 

Although it is not a unique feature of this book, it is written 
with a substantial number of well selected and appropriate figures 
and tables. Illustrative of these are: (1) A Pupil Profile Chart of 
the STEP achievement test, (2) A table of Educational Objec- 
tives in Communication from Elementary School Objectives by N. 
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C. Kearney, and (3) A Table of Quartiles, Deciles, Percentiles, and 
Percentile Ranks Based upon a Distribution of 360 Raw Scores, 
One who is accustomed to teaching the area of Measurement and © 


Evaluation knows how useful is each of these two tables and the 


figure. Many others of similar value are found in the book. 


The strongest section of the volume is Part II dealing with the i 


Informal Methods of Evaluating Achievement. In five chapters the 
authors in a most lucid manner show how needs are translated into 


objectives which in turn are evaluated as knowledge, understanding, — 
procedures, and products with the teacher’s tests and evaluation in- — 
struments. Good examples of types of tests are given, and the use 


of a wide variety of types is urged. 


A 

Each chapter concludes with a summary and with a well selected 
list of annotated Suggested Readings. Appendix A contains the ma- ` 
terial on statistics which is commonly found in measurement text- i 
books as a chapter of the book. This material is condensed into’ 


eleven pages. There is a very brief presentation containing one 
measure of central tendency, the mean; one measure of variability, 


the standard deviation; the product-moment coefficient of correla- 
tion; and three measures of relative performance, the quartile, decile, — 
and percentile. If this material is all of the statistics learned by stu- — 
dents in the course for which this book is intended, obviously it is 


a very shallow coverage. 

Appendix B contains two and one-half pages of the names of pub- 
lishers and distributors of standardized evaluation instruments, 
Appendices C, D, and E consist of eleven pages of tests with sup- 
plemental data of publisher name, year of publication, age or grade 
level intended, time required for administration, and a reference to 
the most recent review in Buros’ Mental Measurements Yearbooks. 
Following the last appendix but before the index are fifteen pages of 
well selected references listed by chapter and in the order in which 
they were cited in the text. 

One of the least satisfying chapters to read and presumably 
one of the most difficult to write is the one on Standardized Achieve- 
ment Tests. As stated earlier the book is a general text in that it 
is written for students planning to teach in the elementary school 
as well as for those who plan to teach in the secondary school. 
Authors of measurement textbooks designed for both groups of 
teachers do have the problem of deciding how many achievement 
tests at each school level should be discussed and, since there are 
so many such tests published, which ones should be chosen for 
discussion or comment. Any tests selected for inclusion will auto- 
matically eliminate other good ones unless authors mention no 


1Kearney, Nolan C., Elementary School Objectives. New York: Russell 
Sage Foundation, 1953, pp. 102-110. 
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specific standardized tests and say in effect, “Go read Tests in 
Print, or Appendices C, D, and E.” But this approach is not helpful 
to the reader who knows practically nothing about the subject. 
One assumes, therefore, that the authors have selected the tests to 
be mentioned on some basis. Those which were included the re- 
viewer thinks are good. But there are no specific achievement tests 
mentioned in many high school subjects such as Mathematics, His- 
tory, Foreign Language, or Science except as they are included in 
test batteries. Similarly, there are many good elementary level 
achievement tests apparently overlooked. 

The same situation obtains but to a lesser extent with respect 
to the aptitude tests and personal-social adjustment tests. Many 
of these such as the Stanford-Binet Intelligence Scale and the 
California Test of Mental Maturity measure intelligence of both 
elementary and secondary school age children, the age and grade 
levels of the pupils to be taught by the teachers to be using the 
book. But the Minnesota Paper Form Board Test and the Dif- 
ferential Aptitude Tests are for secondary level students and 
adults, Prospective teachers of elementary school children have 
difficulty generating an interest in studying about such tests. 

In using this textbook with a class, the teacher can be satisfied 
with the scholarly and interesting presentation of a growing but 
rather well defined body of subject matter. The authors are thor- 
oughly familiar with the topics discussed and with the best refer- 
ences related thereto. It was necessary for the authors to determine 
what should be the major topics of a course in Evaluating Pupil 
Growth. In making this decision it was necessary to truncate some 
sections in order to give complete treatment to others. This re- 
viewer reacts favorably to their selection of topics to be included. 
He is neither adversely critical nor suprised at the decisions as to 
what topics should receive a brief treatment. The references listed 
a re book are for the use of those who desire more breadth or 

lepth, 
GLENN W. DURFLINGER 
University of California, Santa Barbara 


Test Norms: Their Use and Interpretation by Frank B. Womer. 
Washington D. C.: Association of Secondary-School Princi- 
pals, National Education Association, 1965. Pp. 56. $2.00. 

This small booklet has been written for the purpose of clarifying 
concepts and procedures relative to test norms and norming techni- 
ques. Clearly, it has been written for pupils, parents, teachers, 
administrators, and boards of education rather than for the test 
specialist or for those sophisticated in the intricacies of educational 
measurement. It is written in a practical “down to earth” manner. 
It answers many pertinent questions frequently posed by teachers. 
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The percentile norm is used as the basis for discussion with an 
occasional mention of other types of norms. Generally, the booklet 
assumes little or no previous knowledge of norms by the reader. It 
is written in a language which the typical layman would compre- 
hend, such as referring to norms as a “yardstick.” However, 
concepts such as standard deviation, correlation coefficient, and re- 
liability coefficient not only are undefined but also are evidently 
assumed to be previously understood by the reader. 

Not forsaking rigor at the expense of oversimplification is a prob- 
lem which one experiences when writing in a technical area for lay 
consumption. Dr. Womer has met this problem estimably, with few 
exceptions. One rather notable exception is the treatment of test re- 
liability. He indicates that reliability is synonymous with accuracy 
rather than consistency. Expressions such as “average (median)” 
only serve to confuse readers who assume average to be the mean. 
An admirable, but perhaps questionable, effort was accomplished 
in discussing norms without introducing the concept of validity, 
per se. Error of measurement was developed in terms of sam- 
pling error rather than in terms of a standard error of measurement 
concept. 

A reader specifically interested in gaining a conceptual basis 
for such norms as stanines, grade placements, and standard scores 
would have to obtain such information from sources other than this 
publication. However, for interpretation of norms in general, their 
limitations, applications, strengths, and weaknesses, this publica- 
tion is an important contribution to school staff members who 
utilize standardized test results, The present reviewer recommmends 
this booklet especially to school administrators and board of educa- 
tion members. 

Jerry ©. Gartock 
Los Angeles County Schools Office 


An Experimental Approach to Projective Techniques by Joseph 
Zubin, Leonard D. Eron, and Florence Shumer. New York: 
John Wiley and Sons, 1965. Pp. xix + 645. $13.50. 

It should be said at the start that this volume is probably the 
most important one on projective techniques to appear in recent 
years, and perhaps in many years. It is directed to clinical psycholo- 
gists (at least those who will listen) and to psychologists of the 
general-experimental persuasion. The former will probably damn 
the authors, and the latter will derive tremendous satisfaction for 
having decided on non-clinical research activities, 

The discussion of projective techniques is limited to the Rorsch- 
ach and Thematic Apperception Test (TAT), or to some variation 
of the two most important techniques. This volume is not a clini- 
cal manual to assist the psychologist employing projective techni- 
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ques to arrive at better interpretations of test material. Indeed, 
after reading this book, one may wonder whether there is really any 
basis in psychological fact for writing such a manual. These 
writers have done more than review validity and reliability studies. 
They have systematically pointed out methodological and theo- 
retical flaws and have cited the sheer weight of studies with results 
unfavorable to the projective techniques. Although others, of 
course, have cited such difficulties on more than one occasion, these 
authors do it again with skill and brilliance. 

In essence, the writers have pointed out that users and investi- 
gators of the Rorschach, particularly, have been divorced from the 
mainstream of psychometric and experimental psychology in both 
theory and method. The importance of the external criterion for 
validation is emphasized and the difficulties of the criterion 
problem in personality measurement are stressed. The most damag- 
ing blows to the projective body are twofold: separation of “pro- 
jective” technique theory from Freud’s formulation of “projection” 
and divorcing projective techniques from perception. The latter is 
the more serious of the two because it neatly cuts the thread to 
psychological theory and findings that have given respectability to 
the projectionist. 

The two chapters on perception will be of more than passing 
interest to experimental psychologists. To the clinical psychologist, 
these two chapters are critical because it apparently dissects the 
link between personality and perception. Social psychologists work- 
ing in the area of person-perception will be made to feel a bit 
uncomfortable, too. But for the Rorschacher, it will be dismaying 
to have the authors demonstrate that the Rorschach is not a 
perceptual task and that perception offers little if anything in the 
way of a rationale for the instruments. The TAT, which is dis- 
cussed separately, comes out with a cleaner bill of health than the 
Rorschach, as far as its having a foot in the psychological main- 
stream. 

Having criticized the naiveté of “projective theory” as well as 
the lack of objectivity in research and in application and having 
shorn projective psychology of its links to experimental psychology, 
one might expect the authors to argue for the abandonment of 
the projective concept and method. This the authors come close to 
doing, and one suspects that they would really like to see projective 
techniques quietly interred. Instead of arguing for the abolition of 
the Rorschach and the TAT, they have provided procedures for 
giving these devices some potential respectability as psychometric 
devices. These procedures include a series of rating scales, which 
would avoid many of the serious problems found when an investiga- 
tor correlates Rorschach scores with an outside criterion. Presum- 
ably, these scales will face similar problems in determining reliability 
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and validity now facing the Rorschach and TAT as currently 
“scored.” 

What will be the impact of this book on practice and training in 
the projective techniques? The authors are under no illusion that 
their position is the final nail in the coffin of projective psychology 
or that projective methodologies will soon join (to use their ex- 
amples) phlogiston, marasmus, and spontaneous generation—con- 
cepts abandoned by other sciences. Based on personal experiences, 
the reviewer can say that the clinical psychologist cannot read 
this book without developing a vague sense of uneasiness and 
apprehension. Is it possible that some will enter the tabernacle to 
sneer, but stay to pray? 

Purr HIMELSTEIN 
Texas Western College 


The Rorschach Index of Repressive Style by Murray Levine and 
George Spivack. Springfield, Illinois: Charles C. Thomas, 1964. 
Pp. vii 164. 

This small volume is an impressive example of a projective test 
manual and a provocative work on construct validation. Levine and 
Spivack, two research psychologists, set for themselves the task 
of developing “a measure of individual differences in the proneness 
to repress,” by studying various ideational qualities reflected in 
Rorschach responses. The book presents the psychological “en- 
gineering,” which led to the development of a rationale and a form- 
alized system of scoring for the Rorschach Index of Repressive 
Style (RIRS). 

The authors acknowledge their indebtedness to the writings of a 
number of Rorschach workers for their techniques of clinical anal- 
ysis of significant stimulus qualities, In the initial stages of in- 
vestigation, they recognized that the RIRS does not measure ac- 
complished acts of repression, but it seems to be effective as an 
index of a style of ideation—in George Klein’s sense—based upon 
the analysis of such qualities in Rorschach verbalizations as flow, 
availability, and richness. “As a style RIRS should not neces- 
sarily measure ‘repression,’ but rather it should measure a property 
of ideation which predisposes one towards acts of repression, given 
the conditions for defensiveness.” 

They perused theoretical writings and examined a wide range of 
clinical and experimental data in the process of defining the con- 
struct of repression, in testing several nomological relations, and in 
validating the index against other relevant personality measures. 
Throughout the text they emphasize the limitations in their investi- 
gations and conceptualizations. 

In the first chapter they review the relevant theoretical and ex- 


BOOK REVIEWS 235 


perimental history of the concept of repression. They contend that 
verbal behavior is a basic mechanism of repression. Repression is 
defined as avoidant verbal behavior and as a limitation on idea- 
tional processes. This notion of repression is supported by such ex- 
perimental studies as those on perceptual defense and selective 
recall in the face of unplesant or threatening stimuli. Repression is 
seen also as having a relation to the stylistic cognitive characteris- 
tic of leveling-sharpening. They assert that labeling and categor- 
izing of stimuli are primary functions of cognition. Much of the 
research reported in the book is directed toward clarifying the rela- 
tionship between language and thought: “properties of language 
reflect properties of thought . . . . Repressive style is a consistent 
characteristic of an individual and it is manifested in vague, un- 
elaborated language which is lacking in integration and flow.” 
Rorschach responses are viewed as rather uniquely useful in 
the derivation of an index of repressive style, for they are based 
upon a situation in which the test instructions and the test stimuli 
afford the subject minimal directions for labeling the stimuli. 
Therefore, a subject has no recourse but to respond in terms of his 
own frame of current and past experimential patterns to his intro- 
versive and extroversive orientations as represented by the test 
stimuli. They note that the evolution of Rorschach analysis has 
focused increasing attention on the language of the responses. The 
scoring system of the RIRS develops from the following general 
conception: “The more the verbalization of a Rorschach response 
reflects vague, impersonal and unelaborated thinking, and lacks 
integration and flow of ideas, the more repressive functioning has 
been manifest. The more the verbalization of the response is stated 
in specific, affectively toned terms and is characterized by a con- 
tinued and developing flow of words, the less repressive functioning 


is indicated.” Seven scoring categories were derived: specificity, elab- 
oration, impulse responses, primary process thinking, self refer- 
ences, movement, and organization. Each free association is scored 
to indicate the lack of repression: thus, the lower the RIRS score, 
the more repressive functioning is said to be operative in the indi- 
vidual, The seven scoring principles are described in sufficient de- 
tail to enable the reader to use the index in his own research. An 
illustrative Rorschach performance is included, for which the scoring 
and tabulations are given on a specimen scoring sheet, and the index 
is calculated. 

The chapter on reliability is a noteworthy contribution to the 
psychometric study of projective techniques. The authors report 
studies on scoring system reliability, including inter-scorer agree- 
ment, retest reliability in a broad range of test conditions and 
instructions, different sets of inkblots, and overvarying periods of 
time with a variety of subjects. They conclude that the RIRS is a 
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reasonably stable index and deals with a relatively enduring style 
of response. 

In their explanation of the construct validity of the RIRS, 
they examined the relationship of the index to the Rorschach deter- 
minants of the Klopfer and Holtzman methods. A correlation was 
found between total quantity of verbalizations and the absence of 
repression. To test the question of whether verbosity is the under- 
lying characteristic, the total number of words given by the same 
subjects to the Thematic Apperception Test (TAT) was compared 
with their RIRS; no relationship was found. RIRS scores were also 
compared to TAT Transcendence scores to shed further light on 
the psychological processes tapped by the Index. 

RIRS has a low correlation with Stanford-Binet and Wechsler- 
Bellevue intelligence quotients as well as with need achievement 
measures for males from the TAT. Some sex differences in the cor- 
relation of RIRS with intelligence and achievement tests have been 
found. Developmental studies of the RIRS with data from the Gesell 
Institute and the Ledwith study of Pennsylvania students have 
suggested the value of evaluating the RIRS score in developmental 
terms to assess progression in differentiation of personality dimen- 
sions as well as subtlety and precision in the perception of self 
and others. Several studies are reported which compare RIRS scores 
with data from questionnaires which purport to measure anxiety. 
Clinical data and hypotheses suggesting a positive relationship be- 
tween degree of repression and anxiety motivated the authors to 
study anxiety as a significant dimension in the construct valida- 
tion of the term. Findings are equivocal, not only regarding cor- 
relations with anxiety indices, but also with MMPI indices of re- 
pression, sensitization, and self-ideal discrepancy. These findings 
point to the need for extensive testing of the sex differences in the 
functions reflected in the RIRS. One chapter is devoted to the 
discussion of available data on sex differences. Concurrent validity 
studies have produced either negative or inconclusive results. 

To clarify the relationship between RIRS and subjects’ inner 
versus outer frame of reference, the results of responses to field 
dependence, sensory isolation, and leveling-sharpening experi- 
mental situations were correlatied. Low but positive correlations 
were found between inner-frame-of-reference oriented individuals 
and high RIRS scores in contrast with environmentally oriented 
and low RIRS scores. The clinical validity of RIRS using dif- 
ferentially diagnosed textbook cases as well as other clinical ma- 
terial was found to be generally satisfactory in that RIRS scores 
were in the direction to be expected on the investigators’ hypotheses 
for the various nosological categories examined, 


__ The book which is well edited has an attractive format. It is well 
indexed. 
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In the reviewer’s opinion this book merits serious reading by 
clinicians interested in testing techniques, by psychometrists in- 
terested in applying their discipline to the improvement of clinical 
techniques, and by students eager to learn about projective test 
reliability and validity methodology. 

CHERRY ANN CLARK 
The Meyers Clinic, Los Angeles 


Interpersonal Competence and Organizational Effectiveness 
by Chris Argyris, Homewood, Illinois: Richard D. Irwin, Inc., 
1962. Pp. 285. 

The unintended, often dysfunctional, consequences of rational 
organization plans, rules, and procedures have received attention 
by many investigators, the latest being Argyris in this book. Similar 
in many general aspects, such as the self reinforcing aspects of 
such models, Argyris’s study differs in the particular elements of 
the organization studied. 

Specifically, Argyris agrues that since organizations are planned 
as rationalistic entities, emphasis is placed on the intellectual role 
that participants will play in organizations. In developing the in- 
tellectual role, organization planners tend to define rules, policies, 
and procedures which serve to guide people in their work. Neglected 
in this planning process is the rocognition that participants are 
capable of emotional and irrational behavior—behavior which is 
not legitimized by any aspect of the formal organization plan. 
No mechanisms are therefore specifically included in the organiza- 
tion structure which allow emotional behavior to come into the 
open and be dealt with as a paramenter. Implicit in such an ap- 
proach to organization planning is a set of values which presumes 
that the only effective human relationships is best accomplished 
via rational behavior. 

Argyris used this view of the organization plan to advance the 
following model. To the extent that participants accept and are 
guided in their behavior by the values implicit in the organization 
plan, they will tend to establish a social system with norms and 
defense mechanisms which prevent authentic interpersonal rela- 
tionships and create a low degree of interpersonal competence. As 
interpersonal competence decreases in the organization, there will 
be a tendency toward conformity, mistrust, and dependence upon 
superiors within the managerial hierarchy which permeates the en- 
tire organization. In turn, there will be a decrease in effective 
decision-making, increase in organization defenses, and the creation 
of departmental centeredness and organizational rigidity—factors 
leading to a decrease in organizational effectiveness. 

In an attempt to assess the validity of the model, interviews and 
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observations of the top executive system (18 executives) were 
made in the division home office of a large midwestern corporation. 
In addition to providing evidence to test the model, the interviews 
and observations were to serve as inputs into a diagnosis of the 
executive system. Rather than present any of the data uncovered 
in the study, suffice it to say that the data reported seem to sup- 
port the model. That is, in all respects, the executives had developed 
a social system which reinforced rationalistic behavior and sup- 
pressed emotional behavior in interpersonal relationships, with a 
resulting low degree of interpersonal competence. 

Armed with this preliminary confirmation, Argyris entered into 
a much more difficult project. After observing that interpersonal 
competence was low in the organization, Argyris used T-group train- 
ing to see whether the executives’ value system, and hence their 
behavior, could be changed toward higher interpersonal competence. 
After reviewing the evidence which consisted of observing the T- 
group sessions, questioning the executives, and observing their 
behavior, he concluded that the T-group experience had made the 
executives more aware of the importance of effective interpersonal 
relations and that there was a noticeable tendency toward higher 
interpersonal competence. However, because of limited time for 
observation and in light of inadequate instruments for measuring 
values, Argyris did not conclude that the T-group experience had 
definitely resulted in a change in the value system of the execu- 
tives or that organization effectiveness had increased. The validity 
of these conclusions was reinforced by Roger Harrison’s using in- 
dependent measurements obtained from Kelly’s Role Repertory 
instruments. However, it is difficult for the reader to reach the same 
set of conclusions as Argyris, since the major supporting evidence 
consists of dialogue from the T-group sessions and reports from 
interviews conducted after the sessions. One gets the impression 
that the excerpts included were specifically chosen to support the 
viewpoint of the author even though he makes a disclaimer on 
this possibility. Without the complete dialogue or, even more desir- 
able, without being a witness to the sessions, it is difficult to reach 
any conclusion other than that given by the author. 

As a contribution to the field of organization behavior, the book 
must be divided into two parts. The first part, the model formula- 
tion, presents many penetrating insights into the potential impact 
of a modern organization on its participants. In addition, many of 
the implications derived from the model provide testable hypo- 
theses that can serve as the basis for additional research. Also pro- 
vided is a highly informative discussion of T-group training 
as a mechanism for inducing change in organization behavior. 
The major limitation is the author's failure to provide operational 
definitions for most of the important concepts in his model. Particu- 
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larly noticeable is the vague definition of both interpersonal com- 
petence and organizational effectiveness. 

The second part, consisting of the empirical work, is defective 
and incomplete in several respects. First, the data are represented 
in such a form that prevents a reader’s independent judgement. 
Second, the data represent a very limited test of the model or of 
the ability to change an executive value system using T-group train- 
ing, since just one portion of an executive hierarchy in one firm was 
used. No generality can be attached to the results. Third, the author 
did not have at his disposal any validated instruments by which 
to measure either the changes in executive values or the impact of 
value changes on executive behavior. 

Argyris also noted a tendency among the executives involved in 
the T-group training to revert to their old, undesirable values with 
the passage of time. This “wearing off” tendency of supposedly 
internalized values leaves open an important question. Will the 
executives continue in their reversion to the old (pre-T-group) 
values or will the reversion stop at some point with a resulting 
net change in values? Argyris could not answer this question. 

These empirical defects were all recognized by the author. Their 
citation in this review is not to serve as a criticism. Instead, it is 
to forewarn potential readers that the book does not provide either 
conclusive empirical evidence supporting the model developed or 
data on the success of the attempt to induce change in values of an 
executive system. Instead, it provides limited evidence, which when 
coupled with the model developed, will serve as inputs for further 
empirical testing. 

D. J. LavcHHUNN 
University of Illinois 


Interviewing: Its Forms and Functions by Stephen A. Richardson, 
Barbara Snell Dohrenwend, and David Klein. New York: Basic 
Books, Inc., 1965. Pp.v +- 380. $7.50. 

The material contained in this book was derived from the 
Field Methods Training Program conducted at the Social Science 
Research Center of Cornell University. The book is divided into 
four parts: (1) The Interview as a Research Instrument; (2) Re- 
spondent Participation; (3) The Question-Answer Process; and 
(4) Interviewer and Respondent. 

Part of the content stems from research done by the authors on 
the personality characteristics of sucessful interviewers, methods of 
selection and training, and the effects of various question types 
and their formulation on response quality and respondent partici- 
pation. Studies by other researchers are also cited, but their method- 
ology is not cirtically reviewed. 
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The interview for obtaining data and making judgements has 
been subjected to little research in contrast to the counseling inter- 
view, but a myraid of concepts and practices have accumulated. The 
few studies reported by the authors appear to contradict some 
of the prevalent assumptions and beliefs. For example, they pre- 
sent a study demonstrating that leading questions may enhance 
respondent cooperation and the validity of responses. They con- 
clude that leading questions may, therefore, be useful and result 
in as accurate responses as nonleading questions. They further 
suggest conditions under which leading questions may be desirable. 
However, no empirical data are given to support their recommended 
conditions. 

The authors caution that strong rapport is not always desir- 
able. They give plausible examples for their recommendations but 
present no research data to support their contention. Similarly, 
they offer suggestions as to the appropriate visible characteristics 
an interviewer should have for various situations, but again sup- 
porting research data are lacking. This lack of supporting evi- 
dence is true for most of the recommendations offered by the authors 
for improving interviewing. This shortcoming is not really a con- 
demnation of them as much as it reflects the general lack of em- 
pirical evidence for the folklore of interviewing practices. In com- 
parison, competing books on interviewing techniques cite probably 
even less research evidence for the practices they advocate. 

Interesting suggestions are offered for improving the determina- 
tion of respondent samples, selecting interviewers, formulating 
questions, and obtaining respondent cooperation. However, the dis- 
cursive style of writing makes it difficult to note readily these 
recommendations. The chapters not only seldom contain an intro- 
duction or brief preview of their content, but also do not end with a 
summary and set of conclusions. Consequently, it is not easy to 
read the book despite its relatively easy vocabulary and practi- 
cal content. 

Specific recommendations for interviewing procedures may be 
found more readily in other texts in the field such as Bingham, 
Moore, and Gustad’s How to Interview, Fenalson’s Essentials in 
Interviewing, or Kephart’s The Employment Interview in Industry. 
Nor does this book present a theoretical framework as Kahn and 
Cannell do in the Dynamics of Interviewing. The patient and un- 
harried reader doing field research in the behavioral sciences will 
find the book useful. 

Witam E. COLEMAN 
Coleman & Associates, Los Angeles 
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The Psychology of Learning and Techniques of Teaching by James 
M. Thyne. New York: Philosophical Library, 1963. Pp. 240. 
Although based upon theoretical ideas in experimental psychol- 
ogy, this is a simply-written practical book on the use of learning 
theory in teaching. After building a learning model for analysis and 
discussion purposes, the author devotes roughly three quarters of 
the remainder of the book to how-to-do-it examples that apply to 
various teaching problems. 

Thyne’s working model for learning emphasizes four major con- 
ditions: cue, pilot cue, force, and tie. These terms refer (without 
specific mention of the fact) to concepts such as conditioned 
stimulus, unconditioned stimulus, drive, and reinforcement. It ap- 
pears at first that the addition of these new terms instead of the use 
of conventional ones is unnecessary. However, since the author is 
very careful with his definitions, his terms probably avoid some 
negative transfer effects associated with disagreements about con- 
ventional terms. 

Thyne’s book is of value primarily to practicing teachers rather 
than to researchers. His direct reference to data is brief, and there 
is little to no discussion of controversy among important theoreti- 
cal points. In some parts of the book there seems to be an em- 
phasis on S-R contributions; in other parts, the emphasis is upon 
cognitive viewpoints. No deliberate attempt is apparent to connect 
examples to specific theories even though names such as Tolman, 
Guthrie, and Pavlov are used. 

The major teaching problems discussed (with a chapter for each) 
are habit-training, habit-breaking, explaining, recall, and training 
in skills, As a rule, chapters begin with a definition of a problem 
(i.e what are habits?). The discussion then tends to move care- 
fully to related psychological concepts such as conditioning. From 
that point, an analysis is made of a classical experiment with impli- 
cations for the learning model. With the establishment of the model, 
examples are mentioned with careful analyses as to how they tend 
to function. 

Thyne is thorough in discussing tangential concepts such as moti- 
vation to aid the value of the learning model in teaching. Also, his 
attention to subpoints and their meaningfulness is reviewed with 
the insertion of summary statements and sections. 

The last chapter of the book deals with transfer. The thesis is 
different from what is often provided in textbooks on educational 
psychology. Thyne, who does not desire to separate transfer from 
learning, mentions that the term transfer is now used because of 
historical precedent rather than because of psychological necessity. 

The author has written a book that utilizes some of its own sug- 
gestions in the presentation of subject matter to the reader. It 
should be very useful in programs of teacher education, especially in 
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courses named “advanced educational psychology” in which a sup- 

plemental book on learning is valuable. The book lacks a discussion 

of the implications of research on meaningul verbal learning as pro- 
posed by Ausubel. 

CHARLES M. GARVERICK 

University of Illinois 


The Preparation of Teachers, An Unstudied Problem in Education 
by Seymour B. Sarason, Kenneth S. Davidson, and Burton 
Blatt. New York: John Wiley & Sons, Inc., 1962. Pp. xv + 124. 
$3.95 ($1.95 paper cover). 

As might be surmised from the title of this book, the authors 
consider that meaningful research has not been performed on the 
subject of teacher preparation. In defense of this title, the authors 
recognize that much discussion has been made, but the studies that 
have been involved with teacher preparation have been of a sub- 
jective nature both for hypotheses and for conclusions. It is as- 
serted that empirical studies should be performed in which 
teacher education (the authors call it teacher training) is evalu- 
ated according to what teachers do and should be expected to do. 

In the opinion of the authors, the teacher should be recognized 
as essentially an applied psychologist who is concerned with the 
learning process. The preparation of teachers, therefore, should be 
studied in ways that begin with a detailed analysis of the teacher- 
learner relationship. This concerns the understanding of teachers 
as observers, evaluators, and influencers of behavior. 

A review is made of the controversy on teacher preparation that 
has existed between academic and professional groups. The authors 
use this review as an entry topic for elaborating the idea that 
subject matter knowledge does not insure effective teaching. This 
is not to say that knowledge is considered unimportant; it is 
simply that it alone is not sufficient. Neither, on the other hand, is 
a spirit of inquiry that may be an additional value in some courses 
emphasizing knowledge. Knowledge does not expose prospective 
teachers to problems of dealing with individual indifferences in 
students. 

The remainder of the book describes classroom behavior and 
the preparation of prospective teachers to observe children. It is 
stressed that prospective teachers must have special preparation in 
classroom observation that is available as soon as possible after 
a decision is made to become a teacher. Such prospective teachers 
need help in perceiving their role as teachers. A tentative state- 
ment is made to the effect that the best teachers are able to per- 
ceive accurately greater differences among children. With this 
perceptiveness, they are better able to cope with children’s needs. 
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In summarizing the book, one finds the major contribution to be 
the discussion of the program for handling classroom observation. 
The authors insist upon carefully selected introductory remarks 
before observations are made. They stress the importance of prob- 
lems connected with an observer’s failure to distinguish between 
overt behavior and an inference about covert behavior. More- 
over, observers are not told what to look for, and judgments 
have to be justified in ways other than by the use of instructor-fed 
information. 

In conclusion, the authors of this book are very critical of the 
subjective practices used in studying teacher preparation. It is 
thought that teacher preparation will remain uninvestigated un- 
Jess studies are made of a teacher as a psychological diagnostician 
and tactician, However, the authors themselves have not presented 
a study with carefully-analyzed empirical evidence. Their position 
on teacher preparation, which is very interesting, possesses many 
recommendations that are thought-provoking. Again, however, many 
aspects of these recommendations represent overgeneralized con- 
clusions. If documented with appropriate evidence, the recom- 
mendations could become a much more important contribution to 
education. 

Many college teachers involved with teacher preparation should 
be aware of the ideas expressed in this book. It is doubtful that the 
book will be used as a text by undergraduates in educational psy- 
chology or other courses in teacher preparation. 

CHARLES M. GarvErick 
University of Illinois 


Concepts of Development in Early Childhool Education by Peter 
Neubauer (Editor). Springfield, Illinois: Charles C. Thomas, 
1965. Pp. ix + 149. 

This book is an exciting treatment of the most crucial and recent 
concerns in early childhood education; Neubauer, who is director 
of the Child Development Center, which has a nursery school and a 
clinic for the treatment of children of preschool age, reports the 
proceedings of a two-day Institute given at the Center in Septem- 
ber, 1964. l 

The major purpose of the Institute was to clarify ideas related to 
child development, subsequently the bases for criteria in the assess- 
ment of development. The editor states that the wide gap between 
psychiatry and, what we hope education will do in the socialization 
processes for children, needs to be narrowed. His concern is that 
many children are going by, unnoticed by the teacher, when the 
child actually may need help. “Without closer coordination, we 
may be running the risk of either too much treatment or not 
enough. The key to such coordination is the concept of development 
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on which we have not yet arrived at wide areas of agreement” 

. vii). 
ae papers were presented: (1) “Stages of Intellectual Develop- 
ment and Their Implications for Early Childhood,” by David 
Ausubel M. D. and Ph.D., who represented a medical and educa- 
tional psychological point of view; (2) “Language and Develop- 
ment,” by Lili Peller, who is a psychoanalyst; (3) “Developmental 
Considerations of the Nursery School Experience,” by Irving Sigel, 
who is a research director of Merrill-Palmer Institute; and (4) 
“Death Fear and Climax in Nursery School Play,” by Jules Henry, 
who is an anthropologist and sociologist. 

Stages of intuitive functioning and logical reasoning presented 
by Ausubel was an astute work; it gave much substance to a rigor- 
ous discourse by a reacting panel which consisted of people highly 
knowledgeable in their fields. Ausubel included four types of in- 
tuitive processes which he feels are often ignored by many; if those 
four types were examined as separate phenomena semantic and 
pedagogical clarity might result. He also discussed the similarities 
and differences between adult’s and children’s thinking. 

Biber takes issue with what she feels is an assumption on the 
part of many who seem to feel that it is the job of educators to 
deal with an earlier mean emergence of cognitive functioning of 
young children, Extrapolating the non-cognitive from the cog- 
nitive development of children in their early years is one of the 
dangers of contemporary educators. A kind of equilibrium needs to 
be developed for the normal functioning of young children; im- 
ae of cognitive development can lead to psychological dis- 
orders. 

Peller, in her presentation on language development, indicated 
the circular character of early play: “A motion or a sound pro- 
duced by the infant becomes the stimulus for a new act on his part. 
Because the child has played with himself and answered his self- 
created stimulus, he is more ready to play back and forth with 
his mother. On the other hand, a well-cared for infant is more 
persistent in his self-play, because the mother has played with 
him” (p. 68). The human face is very important in a child’s learning 
language; the child responds with great affect to the mother’s face. 

She also indicates that there is an important area of “contacts 
with the environment, which the child himself initiates, and the 
even huger field of self-stimulation,” (p. 73) which many people 
seem to forget when they discuss early childhood education; the 
tendency is to think either of maturation processes or of instruction 
in relation to children’s learning and development. “Spontaneous 
exploration and circular stimulation may even account for most 
of the learning of young children” (p. 73). 

Sigel’s paper discusses the child’s separation from home to 
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school, the physical world of the nursery school, and its contribu- 
tion to the intellectual aspect of children’s development. Very 
little research has been done on the nursery school examined from 
the points of view of his framework. He does emphasize, however, 
that the nursery school teacher not be considered a parent sur- 
rogate, that children are reality-oriented enough to know she is 
not a substitute parent, that she is a helpful adult. Educators 
need to be aware of this concept, too. 

Children are socialized by social rewards which are given for 
“impulse control, respect for others, achievement striving, and 
reality orientation. In other words, in the nursery school, social 
rewards contribute to the child’s acquisition of role behaviors” 
(p. 92). 

Experimentation with roles occurs in the “nursery school experi- 
ence in two ways: (1) the real-life requirements necessitated by the 
child’s ship from a homebound individual to the school child; and 
(2) dramatic play, either spontaneous, or as part of the teach- 
ing program” (p. 93). 

Henry suggests that underlying anxieties about annihilation 
reveal themselves in children’s enactment of commercialized tele- 
vision’s “ceremonialization of death, fear, hostility,” (p. 123) 
which “provides a spectrum of mass-produced roles which chil- 
dren of nursery school age readily make part of their play, thereby 
making possible a certain kind of special interchange among 
themselves” (p. 123). He feels that since fear is a part of the 
culture, the notion has to be dealt with: “All we can expect to do is 
to enable other types of human potentialities to emerge in the 
process of acting out fear. Impulses to mastery, to protectiveness, 
to love, and to clear reasoning readily emerge in the danger 
situation, and ought to be fostered” (p. 123). 

The tendency to project adult notions into interpretation of 
children’s behavior, overt and covert, has been prevalent in re- 
search. Children’s notions about death, sex, and mastery, are mainly 
translated through the adult perspective, but few research studies 
are available to give us satisfactory data on these important ques- 
tions. 

Even though consensus was not reached, congruent to the major 
purposes of the Institute, the book itself, with its discussions, re- 
actions, and suggestions to the major papers presented is an ex- 
cellent source, rich with ideas, and valid in its most recent trends of 
contemporary and crucial concerns in early childhood. This re- 
viewer recommends heartily that anyone interested in the field see 
this book; not only is it fine reading, but it is also a rich source for 
further research. 

EDYTHE MARGOLIN 
University of California, Los Angeles 
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Psychodrama Explained by Samuel Kahn. New York: Philosophi- 
cal Library, Inc., 1964. Pp. vi + 77. 

This slim volume, unencumbered by reference material, explains 
the nature and purpose of psychodrama. The text is preceded by 
an introduction by J. L. Moreno. The psychodrama is viewed as 
a “form of psychotherapy in which the participants enact or re- 
enact situations that are of emotional significance to them.” Its 
use is not restricted just to individuals who are in an abnormal 
state. Kahn believes that almost anyone can gain insight into his 
problems through this procedure. The psychodrama employs 
verbal communication but also includes various non-verbal means 
of communicating such as gestures, movement, and dancing. 

Kahn examines several sources that have contributed to the 
development of this procedure. Among these are Freud’s “social 
instinct” and Trotter’s idea that a group is a “biological exten- 
sion of all higher organisms.” He also traces Moreno’s contribution 
to the psychodrama. The general contrasting of psychodrama with 
other procedures enables the reader to see how it relates to the 
total therapeutic structure. According to Kahn, one of the chief 
values of psychodrama is the spontaneity which it generates within 
the individual and the group. This spontaneity frees forces that 
enhance the therapeutic process in all who are witness to the scene. 

The book has six chapters or subdivisions. The first two chapters 
provide background material. The third chapter, “The Psychoso- 
matic Picture”, examines the need for medical and psychological 
histories of the participants in the drama. It is based on the pre- 
mise that we all are neurotic to a certain degree. The fourth 
chapter is more than a glossary of terms. It is the examination in 
some detail of the ideas, phrases, words, characters, concepts, and 
techniques that are used by those who employ psychodrama as a 
therapeutic procedure. The fifth chapter provides a meaningful syn- 
thesis of the concepts and techniques of the total process. The last 
chapter gives an example of a psychodrama session in action. 

To a certain degree the book tends to appear “simple” in nature. 
However, it does demand a degree of psychological sophistication 
to read it with meaning. For example, the psychodrama is not 
another role playing technique but a form of depth therapy. This 
subtlety takes a little time to be accepted by the general reader. 
Although Kahn has taken care in “explaining” psychodrama, 
only time will tell whether the procedure will have universal usage. 

Henry KACZKOWSKI 
University of Illinois 


Coming of Age in America by Edgar Z. Friedenberg. New York: 
Random House, 1965. Pp. xii -+ 300. $5.95. 
Among the frequent criticisms of educational research is that the 
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results are more often found on a library shelf under a layer of dust 
than in action. The usual nostrum begins with the need to rewrite 
the research report in the vernacular and to project the results be- 
yond the safe hedging of fiducial limits into specific programs for 
action. Dr. Friedenberg has apparently attempted to meet the pre- 
scription. It will be interesting to observe the results. 

A total of 225 students from nine high schools was interviewed 
to secure their opinions on reasonable solutions to fictional situa- 
tions involving “typical” high school students interacting with the 
organization and the staff of a mythical school. From their re- 
sponses, some of which were secured through a modified Q-sort and 
some through a hermeneutic analysis of the interviews, Friedenberg 
discusses the formation and operation of adolescent values in the 
American high school. 

The initial analysis presents the adolescent as a native under 
colonial government. Programs for the adolescent are not initiated 
for the sake of the adolescent but rather to get him to abandon his 
barbarianism and to acquiesce to the patterns of adult life. By 
definition, the adolescent is inferior. Moreover, in every task as- 
signed to him by the school, whether academic, social, or organi- 
zational, his inferiority is emphasized. Assimilation into adult life is 
achieved at the cost of individual freedom, human dignity, and 
elemental justice. 

The theoretical structure of the research is never clearly pre- 
sented. In fact, the author himself points out that he is supple- 
menting the research with other experiences and conclusions and 
that he is using some of his research data to illustrate his points. 
The definition and dynamics of values as used in the book are 
never specified. Nor do they become apparent from the continuity. 
Opinions and attitudes are often used as synonyms for values. 
Furthermore, the relations to overt behavior or action are not clear. 
However, the book is a popular rather than a scientific report; one 
should not complain about the absence of modern plumbing when 
one intends to live with the natives in the bush. 

Following the presentation of the study, which at times seems 
excessively long on data and short on interpretation, Friedenberg 
summarizes his personal views on the treatment of the adolescent. 
He strongly disapproves of compulsory school attendance; the in- 
ability of adults to design and to operate a workable social order is 
no reason to confine the adolescent until he accepts mediocrity. The 
profession of education must begin to base its authority upon its 
own competence to assist the adolescent in his development rather 
than to rely upon tradition, physical force, Machiavellian coercion 
concealed beneath labels of “psychological” or “guidance” activi- 
ties. The school must aspire to the task of education, which has a 
goal of helping the individual to find meaning in his own life and, 
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to become more sensitive to the meaning of the lives of others, and 
to desist from the dowdy, prosaic, essentially goaless task of pro- 
viding inadequate custodial care for the natives until they are tired 
of being restless. 

The book is an interesting one; it should generate a modest 
amount of controversy. It does not present a pleasing picture of the 
American high school or of its students and its staff. Unfortunately, 
all three of these educational segments have been criticized so often 
that, like the natives under colonial government, they accept their 
lot with little complaint and are advised by the master to be grate- 
ful for the few crumbs lest these also be taken from them. 

Grratp T. Kowrrz 
University of Houston 


Readings for Educational Psychology by Ellis Batten Page (Ed- 
itor). New York and Burlingame: Harcourt, Brace & World, 
Inc., 1964. Pp. xii + 404. 

This book of readings was prepared by Page in consultation with 
Lee J. Cronbach. As such, the readings are especially designed to 
accompany Cronbach’s Educational Psychology, Second Edition 
(Harcourt, Brace & World, 1963). Nevertheless, some of the selec- 
tions are valuable readings appearing after Cronbach’s book. Others 
are taken from sources not appropriate for listing in his text, as 
they are relatively unavailable to students. 

The most logical question to be considered is, “Does the collection 
of readings adequately sample the field?” This question particularly 
applies to educational psychology, since that discipline reflects a 
variety of professional interests, rules of evidence, and methods of 
approach. Page has deliberately chosen some great thinkers from 
many fields—psychology, anthropology, sociology, and philosophy, 
all of whom are, of course, “educators!” The authors include Paul 
Mussen, Else Frenkel-Brunswick, Jackson and Getzels, Havighurst 
and Neugarten, Jerome Bruner, and B. F. Skinner. Whereas no book 
of readings could be expected to include all areas of the field, Page 
has presented the grand issues of the field quite adequately. 

Another appropriate question would be concerned with how has 
the editor combined and arranged the book for maximum clarity 
and meaningfulness. Page lists eighteen subjects with two readings 
per subject for a total of thirty-six readings. Titles of subjects in- 
clude, “How Psychology Contributes to Education,” “Communi- 
cating Knowledge,” “Purposes and Aspirations,” “Judging Per- 
formance,” and the like. Apparently Page has put more emphasis 
upon selecting a good reading list and less emphasis upon selecting 
several readings on a particular topic. However, the loss in depth of 
any particular subject is more than made up by the selection of 
worthwhile articles. 
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As might be expected in a readings designed to accompany Cron- 
bach’s book, the over-all emphasis is toward rigor; the editor takes 
the view that educational psychology is at its best an empirical 
social science. The readings chosen are articles involving manipu- 
lation of variables and quantification of results. If education is to 
be defined as an art and psychology as a science, then it appears 
that this book of readings leans more toward psychology than it 
does toward education. In spite of the “solid” approach to the field, 
some “unscientific” readings of note are included. 

Finally, in the introduction of each subject area, no prefatory 
statement is made by the author; instead a brief headnote describ- 
ing the reading, raising some questions, and making some sug- 
gestions appears at the beginning of the article proper. This volume 
is certainly a refreshing change from some books in education which 
one reads and then wonders what one has read. The editor has 
successfully chosen and arranged the selections such that the grand 
issues in educational psychology are cogently stated and the ma- 
terial is highly factual; yet the readings are not so trivial or over- 
whelmingly statistical that the student becomes dismayed. 

r Pur §. Very 
University of Rhode Island 


ERRATUM 


An error has been discovered in the list of references for the 
article “Acquiescence in the MMPI?” by Leonard G. Rorer and 
Lewis R. Goldberg which appeared in the autumn 1965 issue. 
_ On page 817 the reference to the article by L. G. Rorer entitled 
“The Great Response Style Myth” lists the wrong journal title, The 
article actually appeared in the Psychological Bulletin instead of 
the listed Psychological Review. 
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SINGLE AND MULTIPLE HIERARCHICAL 
CLASSIFICATION BY RECIPROCAL PAIRS 
AND RANK ORDER TYPES 


LOUIS L. McQUITTY 
Michigan State University 


Himrarcurcan Olassification by Reciprocal Pairs is a statistical 
method for classifying people (institutions or other objects) into a 
hierarchical system based on indices of association between them 
(McQuitty, 1964). 

The method selects initial pairs of objects which are indicative 
of types in terms of common characteristics and generally expands 
and improves the types by incorporating additional objects and by 
eliminating characteristics which are irrelevant to the descriptions 
of the types. The method realizes both types of any size and suc- 
cessive levels of classification while at the same time limiting itself 
to reciprocal pairs. 

A pair of objects i and j are reciprocal if 2 is most like j and j is 
also most like 7 (i = j) ; but not if only 7 is most like j, or if only 7 
is most like 7; i does not need to be most like j just because j is 
most like 7 and vice versa. 

Reciprocal pairs have two advantages: (a) the very fact that the 
members are reciprocal is generally some evidence that the classi- 
fication is valid; and (b) every matrix of interassociation between 
objects has at least one reciprocal pair; the highest index in the 
matrix necessarily mediates between reciprocal objects, and the 
matrix may have several or many reciprocal pairs of objects. If In- 
dex 7j is the highest index in the matrix, then 7 is highest with j and 
j is also highest with 7. In this latter case, however, the reciprocity 
may be due exclusively to the fact that the index is the highest 
entry in the matrix. 
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If the successive matrices each yield only one reciprocal pair, the 
resulting classification could be due to chance alone; additional 
evidence of validity is required, such as by a repeated and inde- 
pendent study. If, however, the successive matrices produce several 
reciprocal pairs, this, in and of itself, is some evidence of validity, 
but nevertheless not sufficient for many situations. 

No matter how few or how many reciprocal pairs are found in & 
matrix, the members of each pair are classified together and in the ‘ 
next step of the analysis each pair is treated as a single unit in the f 
classification. 

A new matrix is formed. It is composed of interassociations be- 
tween: (1) objects with objects, (2) object pairs with objects, and 
(3) sometimes object pairs with object pairs if the original matrix 
contained more than one reciprocal pair. 

If pairs of objects with pairs of objects or a single object witha 
pair of objects yields a reciprocal pair, then in the next matrix this 
reciprocal pair is treated as a single unit. In the general case each 
pair is treated as a single unit in the next matrix no matter how ~ 
many objects have been involved in building it up from the analysis 
of previous matrices. 

Of the three kinds of interassociations mentioned above, those of | 
Ttem (1) are available from the original matrix. Those of Items (2) 
and (3) can be computed using the classification assumption (Me- 
Quitty, 1960), the similarity index (McQuitty, 1955a), or the cor- 
rected agreement score (McQuitty, 1956). 

The classification assumption assumes that the interassociation 
between two categories is equal to the smallest index when every 
object from among the two categories is paired with every other one 
and their indices computed. ` 

The similarity index uses the indices for all pairs obtained when 
one object of each pair must come from each category; it is the 
average of the indices for these pairs. 

The corrected agreement score is designed to correct the agree- 
ment score, nij, for chance. 


Hierarchical Classification by Reciprocal Pairs Using the 
“Pure-Type” Assumption 


The Assumption 
A difficulty with the above method is the complication involved 
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in computing indices of association between pairs or between an 
object and a pair of objects. 

We have developed a new and simpler version of Hierarchical 
Classification by Reciprocal Pairs. It is based on several assump- 
tions: (1) Every object is an “imperfect” type as distinct from a 
“pure” type; only “imperfect” types exist in reality, and “pure” 
types exist only in theory. (2) There are fewer “pure” types than 
“imperfect” types; each “pure” type is represented in reality by two 
or more “imperfect” types. (3) The characteristics of “pure” types 
are approached but never quite realized by classifying “imperfect” 
types into internally consistent categories, and determining their 
common characteristics; the validity of representation of a “pure” 
type increases as the number of “imperfect” types representing it 
increases. (4) “Hierarchical” types include all of the types realized 
in classifying “imperfect” types into larger and larger, internally 
consistent categories; they are the types intermediate between those 
of reality and theory, “imperfect” and “pure”. (5) Deficiencies in 
the validities of “imperfect” and “hierarchical” types as represen- 
tatives of “pure” types are insufficient to cause erroneous classifi- 
cation when the method of classification is based on the isolation 
of internally consistent categories, as in the case of Hierarchical 
Analysis by Reciprocal Pairs. It is unnecessary to correct for 
chance errors in the indices between “imperfect” types as we clas- 
sify them into larger and larger, internally consistent categories in 
approaching “pure” types. 

The steps in Hierarchical Analysis by Reciprocal Pairs, using the 
“Pure-Type” Assumption, are as follows: 

(1) Compute an index of association between every “imperfect” 
type with every other “imperfect” type to yield a matrix of interas- 
sociations between “imperfect” types. 

(2) Isolate the reciprocal pairs. 

(3) Specify the common characteristics for each reciprocal pair. 
These define a “hierarchical” type. It is now possible to compute an 
index of association between either, (a) two “hierarchical” types, 
or (b) between a “hierarchical” type and an “imperfect” type. 

(4) Substitute the “hierarchical” types for the “imperfect” types 
which produced them. 

(5) Prepare a new matrix of interassociations between types, 
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using only the retained “imperfect” types and the “hierarchical” 
types which replaced the discarded “imperfect” types. 

(6) Isolate reciprocal pairs. 

(7) Continue these steps until the analysis is completed. 


An Illustration 
The method is illustrated in analyzing Table 1, which shows 
TABLE 1 
Agreement Scores between Companies 


A 2 16 16 14 6 11 7 
B 29 17 17 13 6 8 10 
Cc 16 17 26 10 8 9 13 
D 16 17 10 12 11 11 
E 14 13 10 10 21 17 13 
F 6 6 8 12 21 19 17 
G 11 9 11 17 19 24 
H 7 , CUN 11 13 17 24 


Note—Data from McQuitty, 1954 


agreement scores between eight companies in terms of their union- 
management relations as assessed in terms of 32 dichotomized vari- 
ables. Two companies agree on a variable if they are either both 
above or below the median, but not if one is above and the other is 
below. 

The entries for the reciprocal pairs are underlined; they are AB, 
CD, EF, GH; two construction companies, two trucking companies, 
a grain processing and a metal products company, and two gar- 
ment manufacturing companies, respectively. 

As the entry between A and B shows, these two companies have 
common answers on 29 of the 32 variables; they disagree on three 
variables, 

In the next step of the analysis, each “hierarchical” type, such as 
AB, is assessed on,the items on which its two members agree, ex- 
clusively. This means that Types AB, CD, EF, and GH were as- 
sessed on 29, 26, 21, and 24 items respectively in the next stage. 

The next step was to prepare a matrix (Table 2) reporting the 
agreements between Hierarchical Types AB, CD, EF, and GH. In 
order for two types such as AB and CD to agree on an item both 
types must be assessed on that item and all companies of the two 
types would have to have the same answer on the item, all either 
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TABLE 2 
Agreement Scores between Hierarchical Types 


AB cD EF GH 
AB 13 4 5 
cD 13 4 6 
EF 4 4 10 
GH 5 6 10 


above or below the median but not some above and others below. 

The entries for the reciprocal pairs, underlined, show that Hierar- 
chical Types AB and CD join to form a higher level Hierarchical 
Type ABCD, and likewise EF and GH to form EFGH, to yield the 
results shown in Figure 1. 


ABCD EFQH 


Hierarchical Types 


A D 
Imperfect Types [\ H 


Figure 1. Types of Companies in Terms of Some Union-Management 
Characteristics 


It is a characteristic of the data and not the method that the 
classification is by twos. The method requires only that the recipro- 
cal pairs of the first matrix be by twos. Other “hierarchical” types 
can contain any number of members from three up; a “hierarchical” 
type of two members from the first matrix, for example, could join 
a single “imperfect” type of the second matrix. 


An Interpretation 


The example illustrates how the method conforms to its design 
and improves on the validity of the types as it proceeds. This fact is 
illustrated with respect to Type EFGH. In Rank Order Typal 
Analysis, where a test of internal consistency was applied, this cat- 
egory of four companies failed to qualify as a type. (McQuitty, 
1963). In the present study, on the other hand, it did qualify even 
though a test of internal consistency was applied. Tt did so because 
each Type EF and GH was improved as a representative of Type 
EFGH, and the test of internal consistency was applied to them 
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rather than to the individual companies (the “imperfect” types) 
as in Rank Order Typal Analysis. The Types EF and GH were 
improved by eliminating from their respective descriptions items 
on which their two members did not agree. The test of internal con- 
sistency was satisfied in the sense that the two types EF and GH 
produced a reciprocal pair, which is a less stringent criterion than 
used by Rank Order Typal Analysis. 


The Principle of Maximum Classification 


Additional assumptions can be used to (a) increase the qualifi- 

cations of “hierarchical” types in order to qualify as “pure” types, 
* and (b) assist in developing multiple “pure” type methods. 

“Pure” types are assumed to be most adequately approached 
when the number of objects classified into a category times the 
number of characteristics on which they agree is maximized. If this 
product is as large (or larger) for a proposed category than for any 
“hierarchical” types entering the proposed category, then the pro- 
posed category is retained; the analysis proceeds using the pro- 
posed category as a “hierarchical” type. 

Otherwise the proposed category is rejected; the “hierarchical” 
types which were to enter it remain and continue in the analysis as 
two separate “hierarchical” types. 

In order for the two separate “hierarchical” types to classify 
elsewhere in the subsequent analysis, each of them must be allowed 
to qualify as reciprocal with some other type. This is done by ig- 
noring for the balance of the analysis the high index which exists 
between them. Hither of them, ¢ or j, is said to be reciprocal with 
some other type, say k, if either i = k or j = k, except for ij. 

The logic of the above approach is this: some reciprocal pairs 
probably occur by chance alone. Consequently, another check is 
sometimes needed to determine whether or not reciprocal pairs do 
in fact reflect “pure” types. Consequently, they are assumed to form 
“pure” types, if, and only if, in addition to being reciprocal, they 
satisfy the principle of maximum classification as outlined above. 

The principle of maximum classification is not generally applied 
when either two “imperfect” types or an “imperfect” type and a 
Pretest type yield the proposed category; it is assumed that 

n “imperfect” type might win over a proposed category due pri- 
bey to excessive, irrelevant characteristics. Once “imperfect” 


LOUIS L. McQUITTY 259 


types have joined to form a “hierarchical” type the number of irrel- 
evant characteristics has usually been reduced, and generally to a 
considerable extent if there was originally an unusually large num- 
ber of them; erroneous decisions by the principle are therefore no 
longer anticipated. 


Multiple Hierarchical Analysis by Reciprocal Pairs for the 
Isolation of Independent Types 


The principle of maximum classification can be used to convert 
Elementary Hierarchical Classification by Reciprocal Pairs into a 
multiple classificatory method. 

Assume that the principle has been applied as outlined above and 
that it was not satisfied, then the proposed category is rejected. 
Furthermore, the two “hierarchical” types which were to form it 
become “finalized.” The common characteristics of each such “final- 
ized” category are assumed to be the best possible, even though 
somewhat imperfect representatives of a “pure” type. 

Both the finalized categories and the objects which compose them 
are continued in the analysis, but they are first separated com- 
pletely in terms of the characteristics which describe them. Each 
finalized category is defined exclusively in terms of the common 
characteristics of the objects which compose it; each of these ob- 
jects, on the other hand, is defined exclusively in terms of the char- 
acteristics not common to the objects which make up the category: 

As a consequence each reduced object has a zero agreement with 
the finalized category in which the object was originally classified. 

As a consequence of this procedure, both finalized categories and 
the reduced objects can classify in a later analysis. 

The above principle of maximum classification was developed as 
a component part of the development of Multiple Classification by 
Agreement Analysis (McQuitty, 1955b, 1957, 1962; Hemingway, 
1961). This is but one principle, others might be developed and 
shown to be better for specified purposes. ‘ 


An Illustration 


Multiple Hierarchical Classification by Reciprocal Pairs for the 
Isolation of Independent Types can be illustrated with the data 
reported in Table 1. In fact, with this data, the procedures of this 
method and the earlier method are identical for the analysis of 
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Tables 1 and 2; the unique features of the current method are in- 
troduced only after Table 2 has been analyzed. 

The principle of maximum classification is applied to the two 
“hierarchical” types yielded by Table 2, Types ABCD and EFGH, 

The members of Type ABCD agree on 13 items. (Table 2) and 
there are four members. Consequently, the classification capacity is 
13 X 4 = 52. The classification capacity of one of the two types 
out of which they developed is larger than this number. The mem- 
bers of Type AB agree on 29 items (Table 1) ‘and this two-member 
type therefore has a classification capacity of 29 X 2 = 58. Anal- 
ogously, the classification capacity of Type CD is 26 X 2 = 52. 
In terms of the principle of maximum classification, Type ABCD is 
rejected in relation to Type AB. f 

Analogously, Type EFGH is rejected in relation to each Type EF 
and GH. , 

The finalized categories of the first analysis are AB, CD, EF, and 
GH. The description of each A and B was reduced by their 29 com- 
mon characteristics leaving the category, AB, described in terms of 
these 29 items and each reduced object, A’ and B’, described in 
terms of the three remaining items (on which they disagree). The 
other three categories were treated in the same fashion. Using this © 

revised data the agreement scores for the matrix of Table 3 were 
“determined. 
* As Table 3 shows, there is little left here on which to have addi- 
tional classifications; the first analysis was relatively exhaustive 
for the data of this study. 

The Table does illustrate that the method would continue to work — 
even with this data. All of the proposed classifications between any _ 
two types, such as CD and GH would be rejected by the principle 
of maximum classification. This would continue until the agree- 
ment scores of three in the lower right one-quarter of the matrix, — 
mediating between reduced objects, were the highest entries in the _ 
matrix. They would then yield the additional classifications: A’G’, 
BH’, CE’, DF’, D'O’, PO , and F’H’. Even these classifications 
(based on few characteristics) hold a consistency with the first 
classification. In the first classification A and B formed a type as 
did also G and H, and here A’ Joins G’ and B’ joins H’. This same 
type of relationship exists in all of the new classifications except 
for D’G’. 


: 
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TABLE 3 
Agreement Scores between Finalized Categories and Reduced Objects 

RECD ER? GH TAT PB CD! EA VE NG! He 
AB i 4 5 0 0 2 2 8 1 5 2 
CD fa 4 6 0 1 0 0 3 3 1 3 
EF 4 4 Seri ON KOly ey) Ol KL Baie S 
GH ESE hue ed TE ENA S wale pe) ia 2) OF 0 
A’ Di ONL 0. Oe wpe. ets U RAI, OEA VE 0 
B Ole ap Ode ENOAR E Ty bial howell, aie 18 
Cc’ 2 0 0 1 1 1 0 3 1 1 2 
D’ ARO AR T S AEO LUSH BIL 
E LNE, IEN ES NET: iin WNA AN. U. A TALATA 
F Teas MEA TANE r E EA E: EDS: 20 PRE 
G Boo AE R A aa Ua D aN AE 0 
H’ SHS UL N ARA A mone: a ONSA T AS 


* Rejected earlier by principle of maximum classification 


Other Possible Outcomes 


The above outcome need not have ensued with a larger set of, 
data. Suppose we had had the set of data shown in Table 4, which 
incorporates Table 2. Types ABCD and EFGH would be rejected 


just as reported above. 


TABLE 4 
An Expansion of Table 2 to Include a Hypothetical Type IJK 
AB CD EF GH WK 4 

AB 13 4 Be b 
CD 13 4 6 9 4 
EF 4 4 10 8 

GH 5 6 10 7 

WK 12 9 8 7 


With Type ABCD rejected, Type ABIJK becomes a candidate, and 
it qualifies in relation to Type AB; it has a classification capacity 
of 12 X 5 = 60, as compared with the capacity of 58 for AB, as re- 
ported above. Assume that the classification capacity for Type IJK 
is 42 (3 X 14 = 42). Category ABIJK would have been accepted 


as a type. 
Multiple Hierarchical Analysis by Reciprocal Pairs for the 
Isolation of Intersecting Types 


Hierarchical Classification by Reciprocal Pairs always yields a 
typology, a hierarchical system of types of objects, with each type 
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defined in terms of the characteristics which its objects have in 
common. In addition to this predominant typology, there may be 
other typologies which are obscured by the dominant one, but 
which nevertheless can be isolated by appropriate procedures. i 


An Illustration 


Consider, for example, the typology shown in Figure 1. The 
characteristics common to any one of the “hierarchical” types, say 
CD (two trucking companies) might prove to express itself in the 
“imperfect” types and there yield a typology somewhat different 
from the predominant pattern. This possibility can be examined 
as follows. 

Determine the items on which Companies C and D, the members 
of Type CD, agree, and using only these items compute a matrix of 
agreement scores for all companies as shown in Table 5. Since 
Companies © and D agree on only 26 of the 32 items, Table 5 is 


TABLE 5 


Agreement Scores between Cinipanies Using Only the 
Items on Which C and D Agree 


A B Cc D E F G H 
A 25 13 13 10 5 8 
Net 25 14 14 9 4 7 
Cc 13 14 26 7 7 tb 
D 13 14 26 7 7 7 
E 10 9 7 d 19 14 12 
F 5 4 t 7 19 14 14 
G 8 7 $ 7 14 14 22 
H 6 7 9 9 12 14 22 


based on only these 26 items and the highest possible entry in the 
matrix is 26. 
The data of Table 5 were analyzed by Hierarchical Classifi- 
` cation by Reciprocal Pairs, using the “Pure-Type” Assumption. 
The analysis yielded the same types as the analysis of Table 1 ex- 
cept for one important difference. Both analyses yielded Types AB, 
CD, EF, GH, ABCD, and EFGH, as shown in Figure 1. However, 
in the case of Table 1, both ABCD and EFGH, failed to qualify as 
types in terms of the principle of maximum classification. In the — 
ease of Table 5, Type ABCD qualifies and EFGH continues to dis- 
qualify, i.e., in terms of the additional requirement of the principle 
of maximum classification. 
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In the last analysis, Types AB, CD, and ABCD agree on 25, 26, 
and 13 items respectively, yielding classification capacities of 50, 
52, and 52 respectively, and thus barely qualifying Type ABCD. , 

Types EF, GH, and EFGH agree on 19, 22, and 10 items re- 
spectively, yielding classification capacities of 38, 44, and 40 re- 
spectively; Type EFGH fails to qualify in terms of the principle 
of maximum classification. 

The items of all other pair-wise types were each utilized in an 
identical fashion as just reported for those of Type CD. They all 

_ produced the same structure as shown in Figure 1, except the items 
of Type AB; they conformed except that they failed to yield Type 
EFGH. In none of these other three cases did either Type ABCD or 
EFGH qualify as a type in terms of the principle of maximum 
classification; only the items of Type CD (trucking) resulted in 
Type ABCD qualifying in terms of both internal consistency and 
the principle of maximum classification. 

The items of Type CD, EF, and GH each produced Type EFGH 
with a classification capacity of 40, but always with a failure to 
qualify in terms of the principle of maximum classification. 


An li nterpretation 


The typology reflected by Type CD (trucking) is more represen- 
tative of Type ABCD than is the typology reflected by Types AB 
(construction), EF (grain processing and metal manufacturing), 
and GH (garments), and is equally as effective as these other pair- 
wise types in producing Type EFGH; of the four pair-wise types, 
Type CD contains the core most representative of the entire typol- 
ogy as is outlined in Figure 1. 


Hierarchical Classification by Rank Order Types 


Hierarchical Classification by Rank Order Types (McQuitty, 
1963 and 1964) is an elaboration of Hierarchical Classification by 
Reciprocal Pairs (McQuitty, 1964). Reciprocal pairs are internally 
consistent categories of only two objects; each object of every cate- 
gory is more like the other object in the cateogry than it is like any 
object of any other category. 

Internally consistent categories can, however, be composed of 
more than two objects. In this case, every object of every category 
is more like every other object of that category than it is like any 
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object of any other category. Hierarchical Classification by Rank 
Order Typal Analysis isolates all of the internally consistent cate- 
gories of each matrix, irrespective of the size of the categories. 

There is but one exception to this latter principle. The method 
will not accept Category X of x objects as satisfying the definition 
of internal consistency if X includes any x — y objects (y less than 
x by at least 2 or more) which do not satisfy the above definition, 
In other words, the categories must not only be internally consistent 
to qualify as types, but in addition all of the sub-categories which 
were considered in building them must also qualify. If this were not 
true, the method could prove to be extremely laborious with some 
sets of data. 


A Comparison of Analysis by Reciprocal Pairs and Rank 
Order Types 


An analysis by Rank Order Types is not necessarily more rapid 
than by Reciprocal Pairs just because the former method can often 
achieve a solution by the analysis óf fewer matrices; each analysis 
of a matrix by that method is usually much more elaborate than by 
Reciprocal Pairs. In fact, the latter method is generally less labo- 
tious, except for highly-structured data. 

Another advantage of the latter method is that it is presumed to 
improve “imperfect” types and “hierarchical” types very gradually 
as it proceeds. The former method attempts to improve them in 
terms of larger units and can in fact take such large steps that some 
“imperfect” types and even some “hierarchical” types might be 
left out of possible improvements. The less well-structured the data 
the more apt this is to occur. 

Analysis by Rank Order Types is preferable for highly-structured 
data and by Reciprocal Pairs for less well-structured data. 


Summary 


This paper outlines definitions of “pure,” “imperfect,” and “hi- 
erarchical” types and uses them to improve Hierarchical Classifi- 
cation by Reciprocal Pairs and expand the method to isolate mul- 
tiple typologies of both independent and intersecting types. 
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COMPARISON OF TWO APPROACHES—JAN AND 
PROF—FOR CAPTURING RATER STRATEGIES! 


ROBERT J. WHERRY, Sr. ann JAMES C. NAYLOR 
The Ohio State University 


In an earlier paper the present authors (Naylor and, Wherry, 
1965) have described the use of the Bottenberg-Christal JAN tech- 
nique (Bottenberg and Christal, 1961) for the isolation of rater 
policies. In that paper we reported in detail the JAN outcome for 
one of four air force specialties analyzed. Actually four specialties 
were studied and the present paper is based upon the complete 
study. The four specialties consisted of two supervisory levels of a 
mechanical specialty and two versions of an administrative (house- 
keeping) type of specialty, both versions being at the same super- 
visory level. 

The JAN technique is based upon defining the capturing of rater 
policy as the extent to which one can predict the actions of a rater 
from the human characteristics of the data he is being required to 
evaluate. With this definition we have no disagreement, but the 
process of capturing policies and of identifying (describing) them ` 
and similarities and differences among them are two isis 
different problems, as we stated in the earlier paper. 

The JAN technique may tell us ‘that raters 1, 18, 22, and 47 us use 
the same strategy, indicating that the same regression equation 
applied to the stimuli works almost as well as would four separate 
regression equations in predicting the rank order assigned to the 
tatees by these four raters. Raw score regression equations are, 
however, notoriously difficult to interpret, due to such internal in- 

t This research was supported in part by the Air Force Systems Command, 


Contract AF41 (609)-1596. The opinions expressed in this paper are those of 
the authors and not necessarily those of the Air Force. 


267 


268 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


teractions between predictors and criterion as are reflected in sup- 

pressor and other similar effects. Obviously one could revert to 
~ direct comparison of the validity profiles of the raters but with 22 

(or 23) points on each profile and the probability of complex pat- 
_ terns of emphasis interpretation by such inspection could become 
s quite difficult. 

_ It was primarily to enhance the identification of the content and 
differences among profiles that we turned to a factorial approach. 
Tt seemed obvious that the number of factors would be less than the 
number of predictors and that describing rater strategies by means 

-of factor patterns would provide for easier interpretation than trait 
'_. patterns. Hence we decided to do an inverse factor analysis using 
intercorrelations among the raters based upon each rater’s profile of 
validity coefficients as a method to cluster the raters. Accordingly 
four separate factor analyses were carried out using the Hotelling 
at principal axis method with Varimax rotation on the IBM 7094. 
» While using the results of the factor analyses to interpret the 

JAN outcomes, it occurred to us that we had perhaps developed an 
alternate method of clustering raters and capturing their strategies. 
Hence the remainder of this paper will consist of three major sec- 
tions: (1) The outcome of the factor analyses; (2) a Profile of 
Factors (PROF) approach to the capturing of rater strategies; 
and (3) a comparison of the two strategy capturing methods. 


$ Factor Analyses of Validity Coefficient Profiles 


All four specialties yielded four identical factors. Three of the 

» “four specialties yielded a fifth factor. Factors were identified by 
ș selecting raters with relatively pure loadings on a single factor and 
computing their average validity coefficients for each trait, These 

it average validity coefficients were then compared across factors to 
sith identify traits used most (having highest validities) by raters high 
* on @ given factor, the highest coefficient being underlined to indi- 
cate that the trait was to be used to identify that factor. In the 
factor descriptions to follow, the validities for each trait for a given 
factor have been averaged across all samples in which the trait oc- 
curred and in addition the ratio of cases in which it was underlined 
(was the highest value) is also reported. Thus a reported value of 
-550 (2/3) would mean that, for persons with a nearly pure loading 

on that factor, the average validity coefficient across the three 


WHERRY AND NAYLOR 269 


samples in which the trait was used amounted to .550 and that trait 
had the highest validity for these people on that factor in two of 


the three samples. 
Factor A was identified as concern with Leadership on the basis 


of the following average validities: 


Monitors work of subordinates .722 (3/8) 
Willing to delegate work — .684 (2/2) 
Willing to train men — .682 (4/4) 

Able to train others —.677 (1/1) > 
Maintains discipline .688 (4/4) m 
Willing to make decisions .585 (2/2) A 
Supports his men .574 (2/2) 

Gets along with other œ .572 (1/2) 

Able to communicate .550 (2/3) * 
Utilized opinions of subordinates .478 (1/1) », 
Fair and impartial .475 (2/2) 
Informs his men «464 (4/4) 


This factor is seen to include items from the areas of both’ con- . 
sideration and structuring by the leader with slightly higher valid- 
ities for the structuring items. 

Factor B was identified as concern with Compliance on the basis 
of the following average validities: 


= 


Punctual and dependable 784 (4/4) 
Care about details .612 (2/2) 
Care of property and equipment .596 (3/4) 
Willing to learn .559 (2/4) 
Neatness in work .552 (2/2) Å: 
Safe work habits -536 (3/4) 
Good appearance .453 (1/1) 


This factor reflects high evaluation by the supervisor of the careful," 

neat, safe worker who is willing to do what he is told and is willing * 

to learn how to do it. His person, his work, and the work he checks . 

is always up to standard. dea 
Factor C was identified as concern for Job Skill and Knowledge» : 

based on the following average validities: y 


Knows theory of aircraft systems .866 (1/1) 
Knows tech orders, ete. +702 (4/4) 
Plans and organizes work -650 (3/4) 
Able to train others -649 (1/1) 
Proficient in trouble shooting .630 (2/2) 
Able to communicate -612 (3/3) 
Speed and productivity -602 (2/2) 
Recognizes parts and equipment 586 (3/3) 


Utilizes diagrams, ete. .563 (1/1) 
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Workers valued by supervisors high on this factor have ability, 
"knowledge and skill and successfully apply them, 
Factor D is identified as concern with Motivation on the basis of 
the following validities: f 


Plans and organizes work -611 (1/4) 
Willing to learn -598 (2/4) 
Job ingenuity -548 (1/1) 
Perseveres on the job -523 (2/3) 
Performs under pressure -465 (2/4) 
Physical Fitness -266 (2/3) 


“Routine Thoroughness.” Tt is not, however, a very important factor A 
as will become apparent later on. Its high validities were: i 


A 


Careful in routine matters -931 (1/1) 
f Gets along with others «722 (1/1) 
* Thorough in inspections +672 (1/1) 
Satisfactory personal life -498 (1/2) 
Safe work habits -427 (1/3) 


That these factors do indeed explain most of the rater variance _ 
for all four Specialties is shown in Table 1. The Percentage of vari- 


4. The total importance of motivation is fairly equal for the two 
types of jobs but is much more unequally valued at the two versions 
of the administrative task while it is of more ne 
portance for both levels of the skilled tasks, 


5. Factor E makes relatively little contribution for any of the _ 
specialties, 


arly equal im- d 
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6. The first four factors explain over 90 per cent of rater be- 
havior in all four specialties. 


TABLE 1 


Proportion of Rater Variance Explained by Each Factor for 
Each of Four Different Specialties 


Specialty 
se 
Lower Higher Admin. Admin. 


Factor Skill Skill A B 
A. Supervision 42,34 47.34 25.88 29.81 
B. Compliance 18.26 21.79 30.70 22,53 
C. Job Knowledge and Skill 18.26 9.59 14.76 34.99 
D. Motivation 12.39 13.77 18.82 2.44 
E. Persistance (Routine) 1.60 1.74 — 3.30 
Totals: 94.02 94.23 90.16 93.07 


The Profile of Factors Method (PROF) for Capturing Rater Strat- 
egies p 

To implement the factor profile approach as a method for cap- 
turing strategies of raters it is necessary to redefine operationally 
what we mean by capturing a strategy. While JAN used the ability 
to predict what the rater did with the stimuli furnished him, PROF 
will define the capturing of a strategy as the identification of the 
factors (criteria) used by the raters and the relative importance as- 
signed to each such factor. Raters with highly similar factor pro- 
files will be judged to have the same strategies, While admittedly 
the similarity of profiles is hard to judge, some simplifying as- 
sumptions can at least make the task relatively objective. 

In the present application of the method we decided to consider 
primarily three levels of rater involvement. 

1. If a rater had a loading of .50 or more, i.e., had 25 per cent or 
more of his behavior controlled, on a given factor we could consider 
this a major part of his strategy. Such major components will be 
identified by capital letters. 

2. If a rater had a loading between .30 and 49; i.e., 9 per cent up 
to 24.99 per cent of his behavior accounted for, on a given factor 
we could consider this a minor but still important part of his strat- 
egy. Such minor components will be identified by lower case letters. 

3. Loading of less than .30 will, in general, be regarded as in- 
sufficient and be represented by a dot. 
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Thus raters with factor loading of 67, 19, 43, 07, —02 would b 
identified as having a strategy of (A.c..), while a rater with load: 
ings of 03, 54, 63, 32, —09 would possess a strategy of (.B Cd.), 
Raters with identical letter patterns will then be classified as hav- 
ing the same strategy. The nature of each strategy will be already 
predetermined by the letter pattern when the letters stand for the 
previously identified factors and the raters’ level of usage. 

Certain considerations led to these standards. For example, if a 
_ Person is to give major consideration to several factors, we must 

define a major factor low enough to permit such a pattern to emerge. 

Thus with five factors we felt that 25 per cent might be an ap- 
` propriate level. This will allow three (or maximally four) of the : 

factors but not all of them to be considered as major foci of in- 

terest. With fewer factors the level might be set higher, while with 
» more factors it would have to be reduced. Again at the lower level — 
with five factors and average communalities around .90, if each 
factor was given its maximum usage each factor would account for 
18 per cent of the variance. Thus the 9 per cent level (loading of 
30) represents approximately one-half of the maximum possible © 
strength under these conditions. Difference in number of factors 
and average communalities would probably necessitate some other 
standard. The important thing is to adopt some reasonable stand- 
ard and then stick to it in order to keep the process as objective as 
possible. 

Finally, in order to make the method conform to the JAN tech- 
nique in so far as was possible, some attention was paid, in certain 
cases, to the distribution of the remaining loadings of less than .30. 
If a given rater has “insignificant” loadings which are all relatively 
high (+ .19, .23, .28 for example) he will tend to act more like 
taters with “significant” loadings on those factors than will a per- 
son who has all relatively low junk loadings (.04, — .02, .09) or one 
who has all relatively high (but insignificant) negative loadings 
(say —.21, —.24, —.18). Actually this type of consideration turned 
out to be important only in the skill specialties and only when a 
single significant high loading on Factor A was present. 
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Table 2 shows the result of the application of the above stand- 
ards to the four samples. In all, 58 separate strategies emerged, Not 
all of the strategies, however, occurred in any given sample. Actual 
count within the separate specialties indicated the following num- 
bers of strategies: 


Lower Skill 31 


Higher Skill 25 
Administrative A 29 
Administrative B 20 


We seem to have a tendency for more distinct strategies at the 
lower job level in the skill specialty and for the skilled as opposed 
to the administrative specialties. 

Further discussion of the PROF results ¿will be delayed until 
comparison is made with JAN outcome in the next section, 


Comparison of PROF and JAN 


The JAN technique had also been applied to all four samples of 
raters, the results for one specialty have already been reported in 
detail in our earlier article (Naylor and Wherry, 1965). It will be 
recalled that JAN had a built-in F test designed to indicate when 
combination of raters or rater groups into more comprehensive 
strategy patterns should stop. The PROF method simply combined 
all raters with the same upper case, lower case, and dot letter pat- 
tern. Our first comparison, therefore, will be made on the basis of 
the number of strategies reported by the two approaches, These 
were: 


Skill Specialties Admin. Specialties 

Lower Higher A B 
JAN 29 25 25 23 
PROF 31 25 29 20 


These results display considerable similarity. Both methods in- 
dicate more strategies for the Skilled than for Administrative 
Specialties and more strategies for lower level than for higher level 
within the skill specialty. Absolute differences in frequency are 
also quite small (2, 0, 4, 3) in all cases. 

A second basis of comparison might be the variability in fre- 
quency of raters as they are distributed over the various strategies 
discovered. These comparisons are made in Table 3. 
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TABLE 3 
Number of Raters in Strategies Captured by JAN ‘and PROF 


Proficiency 
Skilled : Administrative 
No. of 
Raters in Lower Higher i l ee B 
Strategy JAN PROF JAN PROF JAN PROF JAN PROF 
9 2 1 
8 1 1 
7 1 
6 1 
5 1 2 2 2 3 1 
4 1 2 A 3 fi 1 
3 1 3 3 2 2 1 5 
2 7 5 4° 2 3 8 6 5 
1 18 22 15 16 16 +16 13 7 
Totals: 99 «sl Fad -25 25. 29 23 0 


While the pattern of distribution is not identical for any of the 
specialties it appears that there is more similarity for the skilled 
than for the administrative specialties. Both methods agree that the 
strategies with the greatest consensus among raters occur for the 
higher Skill specialty and the administrative B specialty which 
show the only strategies with as many as eight or more raters in 
agreement, 

Another basis of comparison would be in the factorial complexity 
of the strategies isolated. The number of strategies involving vari- 
ous degrees of complexity are shown in Table 4. Once again the two 
methods tend to show highly similar results with PROF showing a 
slight tendency to discover more complex patterns. 


TABLE 4 
Complexity of Strategies Captured by JAN and PROF 


Number of Proficiency 
Major Skilled Administrative 
and/or + 
Minor Lower ` Higher A B 
Factors JAN PROF JAN PROF JAN PROF JAN PROF 
4 4 5 1 3 5 6 1 1 
3 8 9 9 10 6 10 9 9 
2 12 2 6 7 7 9 11 8 
1 5 5 9 5 7 4 2 2 
Totals: 29, 31 25 25 25 29 23 20 
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Since comparisons of JAN and PROF, up to this point, have ~ 
_ dealt only with average. factor loadings in the JAN strategies, it 
i would still be possible for strategies to have the same “pattern” 
j ‘and even the same number, of raters and still differ in the actual 
raters who were identified as belonging to the strategy. This. final ] 
‘test—consisting “of rater assignment i is dealt with in Tables 5a, 5b, 
50, and 5d. These tables were constructed by using the PROF strat- 
egies: and then extending the number of JAN clusters until the 


ln 
» . 
é bd 
y $ i * 
a f 
Hi * TABLE £ 5a 
i) MAAT X 4 ESM ol ja Strategies Captured by JAN and PROF 
: A : Jor Lower Level Skill Specialty 
„å Pa CODE PROF JAN 
i 
42) AS —— 35 35 
Ay. 14, 23 1, 14, 23 
Ae tt LL44 7, 17, 30, 43, 44 
vee 22 22 
AB oig 24 24 
i aB «ss 31 31 
Ac: 19, 28, 40, 41 40 
ia EOT i: 3, 6, 18, 20, 28, 41 
Avd: 483 11,13 q 
note 5, 10, 16, 29, 30 4, 5,8, 16, 25, 29, 10 
aBC 39 39 
Ab-d- 15 > 1 
aB-D- 3 Ea 
NN T s 
< 2, 12 2, 19, 27, 36 
A-Cd- 6, 20, 0,27 7, 36, 42 2 i 
S 8, 25 x 
c 
abCD of 32, 46 
abeD 46 § oo 
B 26 2 
ie a í 
kd 38 
Éa 50 38 
BD. Bo 50 
-b-D- 33 FA p 
*BcDe 48 48 
Bean 45 45 
HON 9 4 
Number Of Strategies: 31 33 
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greatest agreement between the two systems was achieved. As will 
be seen by inspecting these tables this resulted in using 33, 32, 33, 
and.33 JAN clusters in those tables. 

To test the outcomes, for agreement between the two systems, 
two types of overlap coefficients were computed. Identical raters in 
clusters with the same pattern were underlined and the proportion 
of common raters was divided by 50 (total number of raters) to get 
a Rater Overlap Coefficient. In addition the number of identically 
captured strategies, divided by the total number of different PROF 
strategies for the,specialty is computed and called a Stratégy Qver- 


” 


TABLE 5b +, 
Comparison of Strategies Captured by JAN’ ‘and PROF 
for Higher Level Skill Specialty i 
PROF CODE PROF i RN 
A... 14, 15, 26, 34, 43 14, 15, 26,34, 38,43 + 
1, 3, 11, 30, 36, 44, 46, 49 3, 11, 36, 46, 49; 50, 1730,44 
10, 12, 38, 39, 45 10, 19, 25, 12, 45, 48 , 
5, 42, 48 5, 42, ; 
21, 27 27, 39, 21° 
50 q " 
23 y 22, 23 A 
25, 31 BETE he 
9 a B 
6 fg 28, 6 x . 
20, 28, 32 28, 20, n 
E TE 16 p 
“4 4p sa 
22 A 
29 29 af 
40 40 4 
17 17 A é 
24 24 
8, 13, 33, 35 8, 13, 88, 35 
> : 
9 
T T s 
41 41 
2 2 * 
37, 47 37, 47 
Number of Strategies: 25 32 
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lap Coefficient.: The overlap coefficients based upon Tables 5a, 5b, 


_ 5c, and 5d were: 


Overlap Coefficients 

Rater Strategy 
Lower Skill -70 .87 
Higher Skill .84 88 
Lower Administrative -70 -79 
Higher Administrative 84 95 


_ These coefficients indicate that the two methods arrived at es- 


sentially similar but not identical results. 


As a matter of fact several of the hon-agreements are more ap- 


F * * 


TABLE 5¢ ' * 
Comparison of Strategies Captured by PROF and JAN 


for Administrative Specialty A 
PROF CODE PROF JAN 
Ave. 6, 24 24, 6 
acess 5 5 
Ab-.. 1 
AB 7, 10, 17 1, 10, 28, 1 
an a , 9 , 17 
A-C. 31 31 
AC. 26 26 
A.-D» 25 25 
c 4, 48 4,1 
; ABO. 13 Aa 
‘d 19, 23 ’ 
arb ae i 
s 27, 28, 35, 38 35 
AB-D agus = 
aB-d 36,4: 
Abed aioe hid 
ABed 11, 20, 22,47 11, 16, 33, 20, 29, 27, 38, 48 
aBed 33, 34, 40,45 40, 34, 7, 18, 45, 47, 50 
c 8, 50 8, 12, 46 
aBeD 18, 46 
abCD 14 14, 43 
“Be 2 2, 30, 37 
:b-D Era wae 
“Bed 3 Bi 
-BCD 44 44 
-bcD 43 a... 
‘CD 39 39 
D 2 2 
Number of Strategies: 29 33 
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TABLE 5d 
Comparison of Strategies Captured by PROF and JAN 
for Administrative Specialty B 
PROF 
CODE PROF JAN 
Apsas 10 10 
aB-- 1, 34, 41 34, 1, 41 
Arc 16, 36 36, 16, 49 
AC 11, 15, 23, 32, 38, 11, 23 
aC 30, 35 35, 15, 30 
aE 44 44 
Abe 21, 49 
ABc 14, 25, 33 33, 14, 21, 25 
AbC 9, 13, 19, 28, 29, 38, 39, 42, 45, 48 45, 18, 48, 19, 46, 13, 28, 29, 32, 
39, 50, 9, 42 
ABC:- 2, 6, 12, 37 2, 12, 6, 37, 
aBC.. 17, 43, 46 17 
abC-+ 18, 24 24 
A-c-e 20 20 
aC-e 22 22 
abeD- 4 4 
Bees 3 3 
“BC.- 26, 31, 40 31, 26, 40 
bC- 5,8 5,8 
BCd: 27, 47 27, 47 
z fh 
Number of 
Strategies: 20 33 


parent than real and are concerned with the “insignificant” load- 
ings discussed earlier in setting up the PROF system. For example, 
consider the code [A. C..] for the lower skill specialty 


PROF: 

[A-C] 
3 @ ll @ 26 09 
18 @ OL 13 02 


JAN: [33], Combined at 33 Strategies 
[A-C] 


3 @® 1 @ 2 09 

6 @ N @ @ 04—A-Cd 
18 @ OL 13 02 

20 @ 24 @ @ 01—A-Cd- 
28 @ 07 @ 24 O00—-A-c- 
4 © 14 @ 19 08—A-c 


It might be said that JAN has erroneously added two variables 
with PROF pattern A. Cd. (6, 20) and two from pattern [A. ¢..] 
(28, 41). Actually, however, 3 and 18 do have loadings of 26 and 13 
respectively on factor D while 6 and 20 have loadings of only .30 
and .32. Had the D loadings on 6 and 20 been of the order .03 
lower (an insignificant change) the PROF method would also have 
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combined them. Again variables 28 and 41 have factor loading of 
45 on factor ©, and if these loadings had been .05 higher (a 
probably insignificant change) we too would have combined these 
variables with 3 and 18. Since the boundaries for the PROF ap- 
proach will always be relatively arbitrary, and since the JAN ap- 
proach utilizes all bases for predictive trends we would always have 
many such slight differences in grouping. 

There were some differences between the systems, however, as 
noted above under the discussion of overlap. In order to maximize 
similarity between the two systems it was necessary to extend the 
number of uncombined JAN groups not only beyond the criterion 
levels but actually beyond the number of strategies proposed by 
PROF. This was necessary because JAN refused to combine some 
groups suggested by PROF while combining some raters who were 
obviously quite different from the PROF viewpoint. For example, 
in Table 3 we see that while PROF pattern .B... combines raters 
8, 13, 33, and 35; JAN (at 32 strategies) has them all as separates. 
Their loadings are (from Table 2) : 


8 -19 ®© 10 18 28 


35 2 @ 08 02 —1i4 
> 
JAN did combine 33 and 35 at step 27, did not add 8 until step 14, 
and did not add 13 until step 10 after erroneously combining rater 
13 with rater 18. In the meantime, JAN was making some rather 
peculiar combinations such as: 


Step 38 [Rater 27: ® 8 @ oo -13 
Rater 39: ® @ 8 4 02, 
Step 30 Rater 6: © ® 8&8 @ -2 
Rater 17: @ @ @ @ -03, 
Step 22 {Rater 4: @i 310. es -2 
Rater 14: @ -10 20 27 -09, 
Step 15 [Rater 16: @® @ -oa 2 @ 
Rater 31: © © @ -16 18, 


and the peculiar combination referred to above: 


Step 13 JRater13: —04 ® 10 02 —15 
Rater18: 12 @ -06 @ -06 
In the light of the above it seems that PROF may be a somewhat 
better basis for combinations of raters than is JAN, if the strat- 
egies are to be meaningful and interpretable. 
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It should be pointed out that PROF is closely related to more 
formal measures of profile similarity such as the Cronbach D? 
(Cronbach and Gleser, 1947) and the Cattell rp (Cattell, 1949). 
These two measures are both joint measures of two characteristics 
which serve to differentiate profiles, namely, shape and general 
level. The PROF letter patterns also reflect these same differenti- 
ating characteristics, The capital letter, lower case letter, and dot 
patterns are rather obvious reflections of profile shape since they 
reflect relatively high, medium, and low points respectively. The 
level rules for defining such labels (.50 or above equals capital 
letter, .30 to .49 equals lower case letter; and .29 or less equals a 
dot) plus the definition of a strategy as a group of raters with 
identical letter patterns is designed to assure level distances be- 
tween corresponding profile points will not exceed certain maximum 
values. Thus, the PROF system assures that rp coefficients com- 
puted among raters in the same PROF strategy will be relatively 
high (D? would be correspondingly low). The converse—that raters 
from different strategies could necessarily have low rp values—is 
not necessarily true, however. For example, raters with patterns 
(AbC..) and (aBe..) would be correctly identified as nearly identi- 
cal if the actual loading were (.55, .45, .52, .08, —.03) and (49, 53, 
46, 04, —09). However, even a person using PROF would have 
recognized that these patterns were more alike than those for 
another rater whose letter pattern was, say, (..cD.). Since no 
sampling error statistic is available for difference between rp values, 
the use of such coefficients would scarcely serve as a more objective 
basis for classification (i.e., deciding whether a given rater should 
or should not be included in strategy X or set up as a separate 
strategy). 

Actually to assume that there are any true boundaries which 
sharply separate one rater from another is probably untenable 
since raters can exhibit all shades and degrees of difference and 
similarity along the several dimensions isolated. To isolate regions 
of relative density within the factor hyperspace and to intelligently 
identify these regions is probably all that is required. The exact 
setting of boundaries for these regions so as to include or exclude a 
few fringe cases is probably of little relative importance. We chose, 
therefore, to not actually compute the more formal measures of 
profile similarity or to incorporate them in the PROF process. 
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Wun confronted with the need for comparing relative class 
standing in classes of different sizes, the concept of the Stanine ap- 
pears to be a useful answer. 

The Stanine is a one digit number and occupies only one column 
in an IBM card. Its use assumes that the distribution of marks for 
the various classes have the same mean and standard deviation, 
and are normally distributed. 

Table 1 was designed to convert the relative class standing of in- 
dividual members of classes into Stanine values. To find the Sta- 
nine for a student standing 13th in a class of 37: 


1. Go to the row “Number Ranked” = 37. 

2. Follow across that row to the column which includes the stu- 
dent’s rank 13. 

3. The table entry shows that all of the ranks 9-15 inclusive be- 
long in Stanine 6. 


The table has been constructed to provide Stanines for all pos- 
sible rankings in classes varying in size from 10 to 100. 


z] 
1Copyright 1963, Performance Research, Inc., 1346 Connecticut Avenue, 
N.W., Washington, D. C. 
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TABLE 1 


Ranks Corresponding to Stanine Values for Different Numbers of Things Ranked 


STANINES 
NUMBER NUMBER 
RANKED 1 2 3 4 5 6 7 8 9 RANKED 
10 : 10 9 78 56 H 2 1 10 
1 A 11 %10 8 5-7 ye E ANT 11 
12 ue 12 1011 89 67 45 23 1 12 
13 13 . 11-12, 910 68 45 23 . #1 13 
14 14 -. 213 911 78 46 23 . 1 14 
15 15 l4 13 1012 79 46 38 2 1 15 
16 16 15 13-14 11-12 7-10 56 34 2 1 16 
17 17 16 1415 11-13 810 57 34 2 1 17 
18 18 17 «15-16 12-14 811 57 34 2 1 18 
19 19 18 16-17 12-15 911 58 34 2 1 19 
20 20 19 16-18 13-15 912 68 35 2 1 20 
21 21 20 17-19 1416 913 68 3-5 2 1 21 
22 22 21 18-20 14-17 10-13 69 35 2 1 22 
23 23 22 19-21 15-18 10-14 69 35 2 1 23 
24 24 22-23 20-21 15-19 11-14 610 45 23 1 24 
25 25 23-24 20-22 16-19 11-15 7-10 46 23 1 25 
26 26 24-25 21-23 17-20 11-16 7-10 46 23 1 26 
27 27 25-26 22-24 17-21 12-16 7-11 46 23 1 27 
28 28 26-27 23-25 18-22 12-17 7-11 46 23 1 28 
29 29 27-28 23-26 18-22 13-17 812 47 23 1 29 
30 30 28-29 24-27 19-23 13-18 812 47 23 1 30 
31 31 29-80 25-28 20-24 13-19 812 47 23 1 31 
32 32 30-31 26-29 20-25 14-19 813 47 23 1 32 
33 33 30-31 27-30 21-26 1420 813 47 23 1 33 
34 34 31-33 27-30 21-26 15-20 914 58 24 1 34 
35 35 32-34 28-31 22-27 15-21 914 58 24 1 35 
36 36 33-35 29-32 23-28 15-22 914 58 24 1 36 
87 37 34-36 30-33 23-29 16-22 9-15 58 24 1 37 
38 37-38 35-36 30-34 24-29 16-23 10-15 5-9 34 1-2 38 
39 38-39 36-37 31-35 24-30 17-23 10-16 5-9 34 12 39 
40 39-40 37-38 32-36 25-31 17-24 10-16 5-9 34 12 40 
41 40-414 38-39 33-37 32 17-25 10-16 5-9 3-4 1-2 41 
42 41-42 39-40 33-38 32 18-25 11-17 5-10 3-4 1-2 42 
43 42-43 39-41 34-38 27-33 18-26 11-17 6-10 3-5 1-2 43 
44 43—44 40-42 35-39 27-34 19-26 11-18 6-10 345 1-2 44 
45 44-45 41-43 36-40 28-35 19-27 11-18 6-10 3-5 1-2 45 
46 45-46 42-44 37-41 29-36 19-28 11-18 6-10 3-5 12 46 
47 46-47 43-45 37-42 29-36 20-28 12-19 611 3-5 1-2 4T 
48 47-48 44-46 38-43 30-37 20-29 12-19 6-11 3-5 1-2 48 
49 48-49 45-47 39-44 30-38 21-29 1220 6-11 3-5 1-2 49 
50 49-50 46-48 40-45 31-39 21-30 12-20 6-11 3-5 1-2 50 
51 50-51 47-49 40-46 32-39 2131 13-20 6-12 3-5 12 51 
52 51-52 47-50 41-46 32-40 22-31 13-21 7-12 3-6 12 52 
53 52-53 48-51 42-47 33-41 22-32 13-21 7-12 3-6 1-2 53 
54 53-54 49-52 43-48 33-42 23-32 13-22 7-12 3-6 1-2 54 
55 54-55 50-53 44-49 34-43 23-33 13-22 7-12 3-6 1-2 55 
56 55-56 51-54 44-50 35-43 23-34 1422 7-13 3-6 1-2 56 
57 56-57 52-55 45-51 35-44 24-34 14-23 7-18 3-6 1-2 57 
58 57-58 53-56 46-52 36-45 24-35 1423 713 3-6 12 58 
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STANINES 
LOM I ele a. NUMBEE 
RANKED 1 2 3 4 5 6 uf 8 9 RANKEI 

59 58-59 54-57 47-53 36-46 25-35 1424 7-13 36 12 59 
60 59-60 55-58 47-54 37-46 25-36 15-24 7-14 36 12 60 
61 60-61 56-59 48-55 38-47 25-37 15-24 7-14 36 12 61 
62 61-62 56-60 49-55 38-48 26-37 15-25 814 37 12 62 
63 61-63 57-60 50-56 39-49 26-38 15-25 814 47 13 63 
64 62-64 58-61 50-57 39-49 27-38 16-26 815 47 13 64 
65 63-65 59-62 51-58 40-50 27-39 16-26 815 47 1-3 65 
66 64-66 60-63 52-59 41-51 27-40 16-26 815 47 13 66 
67 65-67 61-64 53-60 41-52 28-40 16-27 815 47 1-3 67 
68 66-68 62-65 5461 42-53 2841 1627 815 47 13 68 
69 67-69 63-66 5462 42-53 29-41 17-28 816 47 13 69 
70 68-70 64-67 55-63 43-54 29-42 17-28 8-16 47 1-3 70 
71 69-71 64-68 56-63 44-55 29-43 17-28 916 48 13 71 
72 70-72 65-69 57-64 44-56 30-43 17-29 9-16 48 1-3 72 
73 71-73 66-70 57-65 45-56 30-44 18-20 917 48 13 73 
74 72-74 67-71 58-66 45-57 31-44 18-30 9-17 48 1-3 74 
75 73-75 68-72 59-67 46-58 31-45 18-30 9-17 48 1-3 75 
76 74-76 69-73 60-68 47-59 31-46 18-30 9-17 48 1-3 76 
77 75-77 70-74 61-69 47-60 3246 18-31 9-17 48 1-3 7 
78 76-78 71-75 61-70 48-60 32-47 1931 918 48 13 78 
79 77-79 72-76 62-71 48-61 33-47 19-32 9-18 48 13 79 
80 78-80 73-77 63-72 49-62 33-48 1932 918 48 13 80 
81 79-81 73-78 64-72 50-63 33-49 19-32 10-18 49 1-3 81 
82 80-82 74-79 64-73 50-63 34-49 20-33 10-19 49 1-3 82 
83 81-83 75-80 65-74 51-64 34-50 20-33 10-19 49 1-3 83 
84 82-84 76-81 66-75 51-65 35-50 20-34 10-19 49 1-3 84 
85 83-85 77-82 67-76 52-66 35-51 20-34 10-19 49 1-3 85 
86 84-86 78-83 67-77 53-66 35-52 21-34 10-20 49 1-3 86 
87 85-87 79-84 68-78 53-67 36-52 21-35 10-20 49 1-3 87 
88 86-88 80-85 69-79 54-68 36-53 21-35 10-20 49 1-3 88 
89 86-89 81-85 70-80 5469 37-53 21-36 10-20 5-9 1-4 89 
90 87-90 81-86 71-80 55-70 37-54 21-36 11-20 5-10 14 90 
91 88-91 82-87 71-81 56-70 37-55 6 11-21 5-19 14 91 
92 89-92 83-88 72-82 56-71 38-55 22-37 11-21 5-10 14 92 
93 90-93 84-89 73-83 57-72 38-56 22-37 11-21 5-10 14 93 
94 91-94 85-90 74-84 57-73 39-56 22-38 11-21 5-10 14 94 
95 92-95 86-91 7485 58-73 39-57 23-38 11-22 5-10 14 95 
96 93-96 87-92 75-86 58-74 40-57 23-39 11-22 5-10 1-4 96 
97 94-97 88-93 76-87 59-75 40-58 23-39 11-22 5-10 1-4 97 
98 95-98 89-94 77-88 60-76 40-59 23-39 11-22 5-10 1-4 98 
99 96-99 90-95 78-89 60-77 41-59 23-40 11-22 5-10 14 99 
100 97-100 90-96 78-89 61-77 41-60 24-40 12-23 5-11 1-4 100 
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SCRIPTIONS ON STATISTICS* ? 


BELA 0. BAKER 
San Francisco State College 
CURTIS D. HARDYCK 
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Tum disagreement between those who belong to what Lubin 
(1962) called the “school of ‘weak measurement’ theorists” and 
those who belong to what might be called the school of “strong 
statistics” has persisted for a number of years with little apparent 
change of attitude on either side. Stevens, as the leading spokes- 
man for the weak measurement school, has asserted (1951) and re- 
asserted (1959, 1960) the view that measurement scales are 
models of object relationships and, for the most part, rather poor 
models which can lead one far astray from the truth if the scores 
they yield are added when they should Only be counted. At least 
two current statistics texts intended for psychologists (Senders, 
1958; Siegel, 1956) present this view as gospel. 

Opposing this view, an assortment of statistically minded psy- 
chologists—e.g., Lord (1953), Burke (1953), Anderson (1961), 

1 This research was supported by research grants from the National Institutes 
of Health, U. 8. Public Health Servicé (MH 07310) and the Research Com- 
mittee, University of California Medical Center. Preliminary work was accom- 
plished by a grant of free computer time by the Computer Center, University 
of California, Berkeley. 

2We are grateful to Professor Jack Block, Professor Quinn McNemar, and 
Miss Mary Epling for their many helpful suggestions throughout this study. 


We are also indebted to Mrs. Eleanor Krasnow who developed and tested he 
computer programs used in this study. 
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McNemar (1962), and Hays (1963) have argued that statistics ap- 
ply to numbers rather than to things and that the formal properties 
of measurement scales, as such, should have no influence on the 
choice of statistics. Savage (1957), a statistician, has supported 
this point of view, stating: “I know of no reason to limit statistical 
procedures to those involving arithmetic operations consistent with 
the scale properties of the observed quantities.” In other words, a 
statistical test answers the question it is designed to answer whether 
measurement is weak or strong. r 

In his widely cited discussion of measurement, Stevens (1951) 
distinguished four classes of scales: Nominal, ordinal, interval, and 
ratio, and specified the arithmetic operations (and hence the sta- 
tistics) which are permissible for each scale. Nominal scales con- 
sist simply of class names and can be treated only by counting 
operations and frequency statistics. Ordinal scales are developed by 
demonstrating that some objects have more of a particular quality 
than do other objects and representing numberically this order 
among objects. Lacking units, the numbers of an ordinal scale 
cannot be added, subtracted, multiplied, or divided, but they can 
be treated by order statistics such as the median or the rank-order 
correlation. Interval scales represent equal increments in the mag- 
nitudes of an object property by equal numerical increments. An 
increase of one unit in any region of an interval scale represents the 
same increment in the object property as does an increase of one 
unit in any other region of the scale. Scores from interval scales can 
be added and subtracted and hence such statistics as the mean, the 
standard deviation, and the product-moment correlation can be 
used. Ratio scales add a true zero point to equal intervals and can 
be multiplied, divided, and treated by subtle statistics which are of 
little concern to most psychologists. 

Although Stevens develops his rationale for relating measure- 
ments and statistics almost exclusively in terms of descriptive 
statistics, he introduces the issue of hypothesis testing somewhat 
obliquely in his discussions of invariance of results under scale 
transformations (1951, 1959). He says, “The basic principle is this: 
Having measured a set of items by making numerical assignments 
in accordance with a set of rules, we are free to change the assign- 
ments by whatever group of transformations will preserve the em- 
pirical information in the scale. These transformations, depending 
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on which group they belong to, will upset some statistical measures 
and leave others unaffected (1959, p. 30).” 

If parametric significance tests, such as t or F are used, the per- 
missible transformations are linear. Only then will invariant re- 
sults be found in comparing groups. An implication of this point of 
view, which is not made explicit by Stevens, is that if a scale is 
viewed as a model of object relationships, then any scale trans- 
formation is a transformation of those relationships. Hence the 
problem of invariance of results under scale transformations raises 
the following question: Can we make correct decisions about the 
nature of reality if we disregard the nature of the measurement 
scale when we apply statistical tests? 

This aspect of Stevens’ position has apparently been ignored by 
many of his critics. Anderson (1961) dismisses out of hand any 
restriction on the uses of ¢ arising from the nature of the measure- 
ments to which it is applied but discusses the question of invariance 
of results under scale transformations seriously and at length be- 
fore concluding that: “The practical problems of invariance or 
generality of results far transcend measurement scale typology” 
(p. 316). 

The aspects of the problem as related to descriptive and inferen- 
tial statistics are as follows: The problem for descriptive statistics 
as presented by Stevens (1951, 1959, 1960) concerns the relation- 
ship of the value of a particular statistic computed on obtained 
measurements to the value of the same statistic computed under 
conditions of perfect measurement. The argument is that the farther 
the measurement model departs from the underlying properties of 
the objects being measured, the less accurate the statistics. In other 
words, this aspect is concerned with precision of measurement. 

In making statistical inferences, however, the issue is whether 
one will arrive at the same probability estimates from different 
types of measurement scales. Given the condition that a measure- 
ment scale may be a very poor model indeed of the properties of the 
objects under study, the question of the effect of the scale on the 
sampling distribution of a statistic remains unanswered. Where 
hypothesis testing is the issue, the appropriate question is: Do 
statistics computed on measures which are inaccurate descriptions 
of reality distribute differently than the same statistics computed 
under conditions of perfect measurement? If not, then a research 
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worker who has nothing better than an ordinal scale to work with 
may have to face the problem of more precise measurement for de- 
scriptive purposes, but at least the probability decisions he may 
make from his ordinal measurements will not be inappropriate for 
parametric statistical models. 

In view of the importance of the issue raised by Stevens for users 
of statistics, it is surprising, as Lubin (1962) notes, that so few de- 
tailed discussions of the problem are available. If Stevens is cor- 
rect, then psychologists should be disturbed about the state of their 
research literature. Since it can be safely asserted that most meas- 
urements in psychology yield scales which are somewhere between 
ordinal and interval scales, many psychologists may have been 
propagating fiction when they have made statistical inferences 
based on significance tests inappropriate to anything less than in- 
terval measurements. If Stevens’ position is correct, it should be 
emphasized more intensively; if it is incorrect, something should be 
done to alleviate the lingering feelings of guilt that plague research 
workers who deliberately use statistics such as t on weak measure- 
ments. 

A test of the issue would seem to require a comparison of the 
sampling distribution of a statistic computed under conditions of 
“perfect” measurement with the sampling distribution of the same 
statistic based on imperfect measurements, Since it is not possible 
to obtain such “perfect” measurements, this comparison is mani- 
festly impossible. As noted above, however, Stevens has suggested 
that the main issue is that of invariance of results when measure- 
ment scales are transformed. Cast in these terms, the problem can 
be examined empirically. All that is required is that the sampling 
distribution of a statistic based on one set of scores be compared 
with the sampling distribution of the same statistic based on scores 
which are not “permissible” transformations of the first set. If 
Stevens is right, these sampling distributions should differ in some 
important way. If they do not, then the nature of the measurement 
scale is, within potentially determinable limits, an irrelevant con- 
sideration when one chooses an hypothesis testing statistic. 


Method 


The statistic selected for study was Student’s ¢. Not only is this 
one of the most commonly used statistics in psychology but it also 
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has the advantage of having been demonstrated empirically to be 
relatively robust in the face of violations of the assumptions of 
normality and equality of variances (Norton, 1953; Boneau, 1960). 

The first set of scores used—which will be referred to as criterion 
(unit-interval) scores—comprised the cardinal numbers from 1 to 
30. Three populations of 1000 scores each were constructed by as- 
signed frequencies to the unit-interval scores to approximate as 
closely as possible the expected frequencies for (1) a normal dis- 
tribution, (2) a rectangular distribution, and (3) an exponential 
distribution (f = 1 + 275° —-5*)). 

According to Stevens (1951), when one uses the mean or stand- 
ard deviation the permissible score transformations are linear. And 
so, to evaluate this proscription, 35 non-linear transformations of 
the unit-interval scores were constructed. These fell into three sub- 
sets, each designed to simulate a common measurement problem in 
psychology. The rationale for this approach is that an investigator 
can almost always develop a measuring device that looks as if it 
yields scores with equal intervals. However, relations among the 
objects represented by these numbers may not be equal—this is, of 
course, Stevens’ main concern—and consequently by producing 
non-linear transformations of cardinal numbers, a class of situa- 
tions is produced directly analogous to the situation that obtains 
when a measurement scale correctly represents the order among 
objects but incorrectly represents the magnitude of differences be- 
tween objects. A statistic such as t, it is argued, cannot be used with 
such a measuring device, since the operations of addition, subtrac- 
tion, multiplication, and division are inappropriate—a condition 
which rather limits the investigator seeking statistical support for 
his conclusions. 

The first set of transformations (1 through 15) was constructed to 
simulate the situation in which an ordinal scale is achieved but 
interval sizes vary randomly. This may often be the case when the 
sum of responses in a keyed direction on a personality inventory is 
used as a trait measure. The first score of each transformation was 
the number 1, to which was added a number selected at random 
within pre-set limits to produce the second score. Another random 
number was then added to the second score to produce the third 
number, continuing until 30 scores had been developed. With this 
technique, the intervals between successive scores could vary ran- 
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domly from 1 to a pre-selected maximum. In order to examine the 
effects of the magnitude of interval variations, three different max- 
imum interval sizes were used in the first 15 transformations. Trans- 
formations 1 through 5 were constructed with a maximum of 2, al- 
lowing the largest interval to be twice as large as the smallest. 
Transformations 6 through 10 were constructed with a maximum of 
10, and transformations 11 through 15 were constructed with a 
maximum of 25. Therefore, in transformations 11 through 15 one 
interval potentially could be 25 times as large as another. Figure 1 
illustrates the relationship between the scores of transformation 7 
and the unit-interval criterion scores. Each preselected maximum 
value was used to develop five transformations—a precaution which 
served to cancel out artifacts of sampling for a particular trans- 
formation, 


TRANSFORMATION 7 SCORE VALUES 


8 16 24 32 40 48 S56 64 72 80 88 96 


-4 6 8 e tf 14 16 18 20 22 24 26 28 30 


CRITERION SCORE VALUES 


Figure 1. Relationship between the intervals of Transformation 7 and the 
unit intervals of the criterion scores. 


The second set of transformations (16 through 25) was devised 
to simulate the situation which may often occur in the measure- 
ment of achievement and ability in which the magnitudes of trait 
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differences represented by intervals at the extremes of a scale may 
be greater than those represented by equal appearing intervals in 
the middle of the scale. Each transformation was produced by 
choosing a number from 1 to a relatively large maximum for the 
first interval, a number from 1 to a somewhat smaller maximum for 
the second interval, and so forth, progressively reducing the max- 
jmum until the center of the score array was reached, then increas- 
ing the maximum at the same rate to the end of the score array. 
For transformations 16-20, the series of maximum values decreased 
by one unit steps from a possible maximum of 15 at the beginning 
of the array to 1 at the center of the array, increasing again by one 
unit steps to a possible maximum of 15 again at the other end of the 
array. For transformations 21-25, the maximum was decreased by 
3 unit steps from 45 to 3 at the center of the array and back to 45. 
The type of transformation produced by this procedure is illus- 
trated in Figure 2. 


TRANSFORMATION 21 SCORE VALUES 
6 56 64 72 80 88 


l EFE SNE 4 å s o 
CRITERION SCORE VALUES 

Figure 2. Relationship between the intervals of Transformation 21 and the 
unit intervals of the criterion scores. 


z2 4 6 


For the third set of transformations (26 through 35), unit in- 
tervals were retained for scores ranging from 1 to 15, the remaining 
15 intervals being varied randomly. This procedure crudely imi- 
tates a problem which may be present sometimes in social distance 
scales or in the Thurstone type scaling of attitude items. For trans- 


96 104 112 120 
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formations 26-30, the maximum interval size above the center of 
the score array was 10. For transformations 31-35, the maximum 
was 25. Figure 3 illustrates this type of transformation. 


TRANSFORMATION 31 SCORE VALUE 
32 53 78 94 108 i8 BO 149 2 178 202 


2 4 6 8 © 2 4 16 18 20 22 24 26 28 
CRITERION SCORE VALUES 


Figure 3. Relationship between the intervals of Transformation 31 and the 
unit intervals of the criterion scores. 


For each of the populations, the unit-interval scores and the 
transformations were stored in the memory of an IBM 7090 digital 
computer as 36 scores for each of 1000 cases, which were identified 
by the numbers 1 through 1000. Random samples of cases were 
selected by means of a multiplying random number generator, with 
some additional features to insure complete randomness and to 
guard against cycling. The generator sequence was initiated by 
obtaining a 35 binary digit (bit) number from the clock of the 
computer. This number was forced to be odd by the insertion of a 
low-order bit. A tape containing the one million random digits pro- 
duced by the RAND Corporation (RAND, 1955) was then refer- 
enced and a word of ten decimal digits read from this tape. The 
first half of the 10 digit word, R, was then transformed by the 
value 8R + 3. This value served as the initial multiplier for the 
number obtained from the computer clock, This multiplication 
produced a 70 bit number, the right 35 bits of which were ex- 
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tracted from the random number. The left bits of the latter were 
used to obtain an integer in the range 1 — 1000. This value de- 
termined the “case” to be selected from the population. The 35 bit 
random number was retained and multiplied by 8R + 3 to obtain 
the next value from the population. 

The second half of the ten-digit number first read from the 
RAND tape was divided by the quantity (NA + NB) /2, where NA 
and NB are the respective sizes of the pairs of samples drawn. The 
remainder occurring from this division determined the number of 
times the value 8R + 3 was used as a multiplier in generating a 
random sequence. When the number of values generated equaled 
the value of the remainder, another ten digit record was read from 
the RAND tape and the sequence of generation repeated. When 
samples of size NA and NB had been drawn, t was computed using 
a standard computing formula. 

A total of 36 ¢ values were computed for each pair of samples 
drawn: One value for the unit-interval or criterion scores and one 
for each of the 35 transformations of the criterion. This is analog- 
ous to sampling from a pool of 1000 subjects with scores on 36 
variables, the first variable representing a set of measurements with 
equal intervals and the remaining 35 variables representing meas- 
urement scales standing in varying non-linear relationships to the 
first set. 

The following notation will be used throughout the paper: 


NRE ERR the type of distribution used: Normal, Rec- 
tangular, or Exponential 

5,5; 15,15; 5,15 ..... size of first and second samples 

ON UR shah yeuies Va the criterion set of values 

a ain sola eed the nth transformation 

Mbp Skilet as ieee. transformations j through k. 


The sequence of the notation is as follows: N,15,15,T¢-10; a nor- 
mal distribution with sample sizes of 15 and 15 for transformations 
6 through 10. 

The computations were summarized in three forms: (1) Con- 
tingency tables showing the relationship of the criterion ¢ value to 
each transformation t value for the .01 and .05 significance levels 
were tabulated. These tables allow the determination of the differ- 
ence in the percentage of t’s reaching given significance levels for a 
particular transformation and for the criterion scores. (2) Fre- 


É 4000 SAMPLES 
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quency distributions of all sets of t values were tabulated. Figure 4 
shows the frequency distributions of t for N, 5, 5, C; N, 5, 5, Ts; 
N, 5, 5, T2o; and N, 5, 5, Tss. 


@ CRITERION 

OTRANSFORM 35 
ATRANSFORM 5 
ATRANSFORM 20 


ogee 
2 O =450 =380 =270 =o =90 © 90 
FIRN VALUE OF # 
Figure 4. Empirical sampling distributions of £ for 4000 pairs of samples 


with NA = NB = 5, from a normal distribution for the Criterion and Trans- 
formations 5, 20, and 35. 


BO 270 350 450 540 630 


(3) Pearson correlation coefficients between the criterion ¢ values 
and each of the transformation ¢ values were calculated over the 
total number of sample pairs drawn for a given condition. The 
standard error of estimate was also computed. These statistics pro- 
vide estimates of the degree to which t's based on transformed 
scores varied from ¢’s based on unit-interval scores for the same 
pairs of samples. 

Parts (1) and (2) of the computation summaries are directly 
relevant to the question of the effect of the measurement model on 
the sampling distribution of t. Part (3) is an attempt to represent 
the deviation of the various types of “inappropriate” measure- 
ment models from a true value (represented by the criterion). 

As has been mentioned earlier, three types of distributions— 
normal, rectangular, and exponential, were used. Three variations 
in sample size were studied—NA = NB = 5; NA = NB = 15; 
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and NA = 5, NB = 15. These combinations are identical with 
those used by Boneau (1960) and permit the present results to be 
compared directly with his. 


Results 


As a first step, the distributions of £ on the criterion measure- 
ments for all conditions were compared with the theoretical dis- 
tributions for the appropriate degrees of freedom. Table 1 contains 
the percentage of ¢ values falling in the 5 per cent and 1 per cent 
regions of the distribution for the criterion scores. 


TABLE 1 


Per cent of t's Based on Criterion Unit Interval Scores Falling in the 5% 
and 1% Regions of Rejection for 4000 Random Sampling Runs 


Sample Sizes 
NA = NB=5 NA = NB =15 NA = 5, NB = 15 
Population 5% 1% 5% 1% 5% 1% 
Distribution Level Level Level Level Level Level 
Normal 4.8 8 5.4 1.0 5.3 1.1 
Rectangular 5.4 1.6 4.6 9 5.1 8 
Exponential 3.9 39 5.1 1.0 4.2 9 


The deviations of the empirical distributions from the expected 
theoretical values for the normal curve are quite small. The results 
are very similar to those reported by Boneau (1960), including the 
underestimates for the exponential distribution. Somewhat sur- 
prisingly, the deviations are reduced only slightly from those re- 
ported by Boneau, despite the fact that the present results are based 
on 4000 t’s as compared to Boneau’s 1000. 

The results for the first set of transformations, which were con- 
structed to simulate a situation where intervals vary randomly 
throughout the range of the measuring instrument, are given in 
Table 2. Since the tabulation of results for individual transforma- 
tions within and across sets Ty-s, Te+o, and Ti1-15 revealed little 
variation, only mean values for all transformations are given in 
Table 2.3 The mean value tabled for each transformation is based 
on 60,000 ¢’s. 

Examination of Table 2 indicates that random variations tend to 
have little effect on the number of ?¢’s falling in the 5 per cent and 


8 Complete tables are available on request from the authors. 
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TABLE 2 


Per cent of t's Falling in the 5% and 1% Regions of Rejection When 
Interval Sizes Vary Randomly (4000 Samples Per Condition) 


5% Level 1% Level 
Total % in Total % in p 
A Larger Tail % Larger Tail 

N,5,5: O 4.8 2.4 8 A 
Tis 4.8 2.4 8 4 

N, 15,15: C 5.4 3.0 1.0 .5 
-15 5.3 2.9 9 5 

N, 5,15: C 5.3 2.7 Ta 6 
Tins 5.0 2.6 2AL 6 

R,5,5: C 5.4 3.0 1.6 9 
Tins 5.3 3.0 1.6 9 

R, 15, 15: C 4.6 2.6 -9 5 
1-15 4.6 2.5 9 5 

R, 5,15: C 5.1 2.7 8 T 
Ti-ts 5.0 2.6 9 ag 

E, 5,5: C 3.9 2.0 9 5 
Tins 3.9 2.0 8 4 

E, 15,15: C 5.1 2.6 1.0 6 
Ti-ts 5.2 2.6 1.0 6 

E, 5,15: C 4.2 3.8 9 9 
Tins 4.2 3.8 1.0 1.0 


1 per cent regions of rejection. Columns (1) and (3) contain the 
total percentages for the 5 per cent and 1 per cent levels of the t 
distribution for all distributions and sample sizes. For the first 
group of transformations, the total percentages in the critical re- 
gions are very close to the theoretical expectation for a normal 
distribution. The largest discrepancy present is for E, 5, 5,—a dis- 
crepancy of only 1.1 per cent. Columns (2) and (4), which contain 
the percentage in the larger tail, show similar minimal variations. 
If one allows for the effects of sampling and takes the deviations of 
each transformation from the obtained percentages of the C dis- 
tribution, the discrepancies become almost nonexistent. The largest 
deviation is .3 per cent for N, 5, 15. It is evident that random vari- 
ations in interval sizes, regardless of the magnitude of those varia- 
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tions, have virtually no effect on the percentage of t’s reaching con- 
ventional significance levels. 

One condition reported in Table 2 (E, 5, 15) did result in an 
asymmetrical ¢ distribution. However, as can be seen by examining 
the tabled values for E, 5, 15, the C distribution is equally asym- 
metrical. For E, 5, 15, the majority of the t values reaching the 5 
per cent level and all of the ¢’s at the 1 per cent level are in one tail 
of the distribution. The direction of the skewing is negative, indi- 
cating that where large differences between sample means occurred, 
the higher mean tended to be the mean of the smaller sample. On 
the basis of this finding, an experimenter would be ill-advised to 
use a one-tailed test when he is using samples of unequal sizes (at 
least if the sizes are of the magnitudes used in this study). How- 
ever, it makes little difference whether there is an interval scale of 
measurement or not. 

The results for the more irregular and extreme transformations 
(16 through 35) are presented in Table 3. Again, since there was 
little variation within sets only mean values for transformations 
16-25 and transformations 26-35 are presented. 

An inspection of Table 3 also permits the conclusion that the 
magnitude of variations in interval sizes has little effect on the t 
distribution. At the same time, it is apparent that t is affected more 
by these types of transformations than was the situation for simple 
random variation. However, the discrepancies are still far from ex- 
treme, In columns (1) and (3) the largest obtained discrepancy is 
23 per cent for E, 15,15,T2s-ss at the 5 per cent level. In columns 
(2) and (4), the largest discrepancy is again at the 5 per cent level 
for E, 15,15,T2¢-ss, a value of 1.1 per cent. 

When compared to the 5.1 per cent of é’s falling in the 5 per cent 
region for the E, 15, 15, C distribution, this discrepancy of 2.3 per 
cent seems rather large. However, it seems slight compared to the 
discrepancies obtained when more serious violations of the as- 
sumptions for the use of ¢ are made, For example, Boneau (1960) 
reported 16 per cent of obtained ¢’s at the 5 per cent level for sam- 
ples of 5 and 15 drawn from normally distributed populations with 
unequal variances. 

When Table 3 is examined for asymmetry, it is evident that the 
transformations in which the intervals in one half of a scale stand 
for substantially smaller variations in the objects being measured 


304 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


TABLE 3 


Per cent of t’s Falling in the 5% and 1% Regions of Rejection 
When Interval Sizes Vary More in Some Regions of the 
Scale Than in Others (4000 Samples Per Condition) 


5% Level 1% Level 
Total %in Total % in 
% Larger Tail % Larger Tail 
Cc 2.4 8 
N, 5, 5: Tie-25 4 
Tas-35 8 
Cc 1.0 
N, 15,15: Thess 6 
Teros5 9 
Cc 1 
N, 5,15:  Tis-ss 1 
Too-ss 1 


m 


Cc 
R, 5,5: Tress 


Treas 


Cc 
R, 15, 15:  Tie_os 
Tao-s5 


Cc 
R, 5,15: Thess 
Teo-as 


Cc 
E, 5, 5: Tiss 


1 
26-35 

Cc 1 

E, 15,15: Tress 1 
Teo-a5 
— 


E, 5,15: Thess 
20-35 


HHQ 

oee woan waw peo paa pao pom meo Pom 
Wan one owo oom OAD Poy Now roy NP OO 
Qo ANQ Qvo HON ORD ONO anna Dao no 
OND Wm RNR HON PRE TOO boao UR POR 


owo PHO NRO Ho www WOM oo 


than do intervals in the other half of the scale—T26-s5;—yield seri- 
ously skewed distributions of ¢ for all conditions where unequal 
sample sizes are used. For E distributions, skewing is present for 
most of the transformations. These transformations provide the 
only situation where the nature of the scale transformation affected 
the sampling distribution of ¢ to a more serious degree than could 
be attributable to the use of unequal sample sizes drawn from an 


BELA O. BAKER, ET AL. 305 


exponential distribution. Even for this condition the effect is quite 
small. For any real-life situation in which the possibility of such 
a measurement scale exists, an experimenter should be chary of 
using ¢ to make a one-tailed test between means based on unequal 
sample N’s. Fortunately, this problem occurs only rarely and when 
it does occur the use of equal sample sizes will minimize the distor- 
tion. 

In reviewing the results presented so far, the following general- 
izations seem warranted: 

1. The percentage of t’s reaching the theoretical 5 per cent and 
1 per cent levels of significance is not seriously affected by the use 
of non-equal interval measurements.‘ 

2. To the extent that there is any influence of the scale trans- 
formation on the percentage of t’s reaching theoretical significance 
levels, the influence is more marked when intervals in one broad 
region of a scale are larger than intervals in another region of the 
scale than it is when interval sizes vary randomly. 

3. If an investigator has a measuring instrument which produces 
either an interval scale or an ordinal scale with randomly varied 
interval sizes, he can safely use ¢ for statistical decisions under 
all circumstances examined in this study. The single exception is 
that t should not be used to do a one tailed test when samples of un- 
equal size have been drawn from a badly skewed population. 

4, If a measurement scale deviates from reality in such a fashion 
that the magnitude of trait differences represented by intervals at 
the extremes of the scale may be greater than those represented by 
equal-appearing intervals in the middle of the scale (Tis-25), it 
seems reasonably safe to use t. Unequal sample sizes can even be 
used if the population is symmetrical, but the proscriptions against 
using one-tailed tests for unequal sample sizes from exponential 
populations still apply. 

5. If the scale is of the kind represented by the relationship be- 
tween C and Tze-ss (in which inequality of units is present in one- 
half of the distribution only), it is still safe to use t, with a some- 
what stricter limitation on the use of one-tailed tests. This arises 

41t is possible that the effects of the scale transformations used in this study 
are actually due to changes in the shape of the distributions which the different 
transformations produced. However, if this is the case, the arguments pre- 


sented regarding the insignificant effects of the nature of measurement scales 
on probability statements are strengthened even more. 
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from the finding that for all population distributions these trans- 
formations yielded skewed distributions of t when unequal sample 
sizes were used. 

6. As a maximally conservative empirical set of rules for using 
t, the following restrictions would seem to be sufficient to compen- 
sate for almost any violation of assumptions investigated up to this 
time: 

a. Have equal sample sizes. 

b. Use a two-tailed test. 

7. Returning to the question as originally formulated: Do sta- 
tistics computed on a measurement scale which is at best a poor fit 
to reality distribute differently than the same statistics computed 
under conditions of perfect measurement? The answer is a firm 
“no,” provided that the conditions of equal sample sizes and two- 
tailed tests are met. The research worker who has nothing better 
than an ordinal scale to work with may have an extremely poor 
fit to reality, but at least he will not be led into making incorrect 
probability estimates if he observes a few simple precautions, 

As a final step, a different sort of analysis will be cited. The prev- 
ious results and discussion related to one aspect of the measure- 
ment problem as posed by Stevens (1951) ; a second aspect remains. 
This concerns the accuracy of the descriptive statistics when the 
Measurement model is a poor fit. Stevens has presented his point of 
view almost exclusively in terms of descriptive statistics and has 
tended to use illustrations from descriptive statistics to support his 
arguments. In the last analysis, this would seem to raise an episte- 
mological question, since it is concerned with the relationship of 
measurement to a true value which cannot be known. However, evi- 
dence as to the correctness or incorrectness of the point of view can 
be examined from the data of the present study, even though the 
results are of no help in solving the problems faced by an experi- 
menter who is wondering how to evaluate the validity and the 
precision of his measuring instrument, 

The question of the accuracy of representation can be evaluated 
by defining the unit interval criterion ¢ values as true measures and 
the values calculated on the various transformations as those ob- 
tained on a measurement model which misrepresents reality. Then 
the degree of relationship between the values of ¢ calculated on spe- 
cific samples for C and the values calculated on Ty-s5 can be ob- 
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tained. This is a correlational question and the results are re- 
ported in Table 4. 

Columns (1), (3), and (5) contain for each of the distributions 
the correlations between values of t for each set of transformations 
and the corresponding values of ¢ for the criterion. The correla- 


TABLE 4 


Meant Correlation Coeficients and Standard Errors of Estimate for 
the Prediction of t's Based on Transformed Scores from ts 
Based on Criterion Unit-interval Scores 


NA =NB=5 NA=NB=15  NA=5,NB=15 


Mean Mean Mean Mean Mean Mean 

Population r By-x T By.x r Byz 
Distribution (1) (2) (3) (4) (5) (6) 
N: Ti-s .997 .089 -997 +082 997 -084 
To-10 .996 =. 111 -995  .100 +995 .104 
Tu-is .992 146 991 138 -991 142 
Tio-20 975 +244 966 -265 970 260 
Toi-25 +968 271 964 274 966 278 
Tro-s0 -935 401 -933 -880 933 .386 
Tass +914 462 911 434 +912 .439 

R: Tizi .999  .056 -988 048 +999 033 
To-10 +996  .094 -996 = .081 +996 084 
Tus 994.117 .994 = .088 +988 104 
Tis—20 .973  .256 -973 .231 -973 +233 
Taas .973  .258 -975 227 976 225 
Tae-20 .948 —.368 943 +339 +944 348 
Tass -927  .430 +922 894 +924 404 
Bis Lins 994 121 -993 113 +992 117 
Ts-10 -992 138 992 -126 992 .129 
Tus 984 199 -985 =. 181 983 189 
Tio-20 -970 .283 +946 842 +951 824 
Tass -963 -3813 .953 314 954 309 
Teo-s0 -981 .218 -930 .382 -940 825 
Tus .966  .288 885 483 +922 405 


s Median values do not differ until the third decimal place for the majority of transformations, 


tions are impressively high. However, because of the broad range 
of values in the ¢ distribution, the standard errors of estimate in 
columns (2), (4), and (6) are more informative statistics. 

Several points can be noted in connection with Table 4: There is 
a regular progression in the size of the standard errors of estimate 
across the sets of transformations used, such that they are smallest 
for Tis, and largest for T2s-ss. These standard errors also become 
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larger as the magnitude of variations in interval size increases, but 
this is less striking than the differences among types of transforma- 
tions. Variations in sample sizes and in the shape of the population 
distribution do not seem to have much influence on the standard 
errors of estimate; consequently these results seem to show a spe- 
cific influence of scale transformations on the values of t. The cor- 
respondence between values of ¢ based on the criterion unit interval 
scores and values of t based on transformations decreases regularly 
and dramatically—from standard errors of estimate on the order of 
-08 to standard errors of estimate on the order of .45—as the de- 
parture from linear transformations becomes more extreme. Here, — 
then, is a finding consistent with Stevens’ expectations: The value | 
of t determined for a comparison of samples of non-interval scores _ 
does tend to be different from the value of ¢ based on interval — 
scores for the same samples and the discrepancy tends to become 
greater as the departure from equal intervals is more marked, 

In conclusion, the views presented by Stevens (1951, 1959, 1960) — 
and by advocates of his position such as Senders (1958), Siegel — 
(1956), and Stake (1960) state that, when one uses t, the measure- d 
ment model should have equal intervals representing linear trans- 
formations of the magnitudes of the characteristics being measured, 
or the statistic will be “upset.” This view may be correct if one 
considers single specific determinations of a statistic in a descriptive 
sense—this seems to be the significance of the standard errors of 
estimate reported in Table 4—but it is incorrect when applied to 
the problem of statistical inference. 

The present findings indicate that strong statistics such as the t a 
test are more than adequate to cope with weak measurements— 
and, with some minor reservations, probabilities estimated from 


the ¢ distribution are little affected by the kind of measurement 
scale used. 
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POWER OF THE ANALYSIS OF VARIANCE OF IN- 
DEPENDENT GROUPS ON NON-NORMAL AND 
NORMALLY TRANSFORMED DATA 


PAUL A. GAMES ann PATRICK A. LUCAS+ 
Ohio University 


Tum science of data analysis, as described by Tukey (1962), (in 
contrast to statistics), has a goal closely tied to the real world: to 
derive certain conclusions about the “state of the world” from a 
sample of evidence. Statistics, on the other hand, is a mathematical 
discipline and can’t be evaluated by the same empirical considera- 
tions. Consequently, although data analysis finds statistics very 
useful, it is helpful to maintain a distinction when approaching the 
practical problems considered here. Two aspects of data analysis 
are relevant to this paper: (1) the choice of the best statistical test 
for the analysis desired and (2) an evaluation of how the test will 
be affected by deviations of the data from the assumptions of the 
test selected. 

In a strictly mathematical sense, the experimenter (Z) maps a 
set of elements which are non-numerical but of conceptual interest 
in psychology onto a set of numbers when he measures a psychologi- 
cal variable. The mathematician would insist that Æ specify (1) the 
number system onto which he maps (such as integers or all rational 
numbers, etc.) and (2) what arithmetic operations and relations 
are implied in the mapping. This second point indicates whether 
order is present and whether the numbers can be added and/or 
multiplied meaningfully. Hs who are “fixated” at this level com- 
monly worry about nominal versus ordinal scales, etc., but their 
worries are rarely useful since little is known about the operations 
and relations that hold for the original variable. 


1 Now at University of Michigan. 
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Knowledge of the number system plus an assumption of how the 
probabilities of the numbers are related gives the mathematician a 
probability density function of the numerical variable—this is the 
basic mathematical model of the statistician. Given this model_ 
(e.g., binomial, Poisson, or normal) the statistician can then spec- 
ify the probability of any admissible statistic describing a random 
sample taken from that population of numbers. Es “fixated” at this 
level worry about the distribution of their sample data—again 
with rare knowledge of what the population really looks like. 

Part of the science of data analysis, then, is to choose the best 
model of the “real world” numbers which have resulted from em- ; 
pirical measurement. Obviously, however, there are only a finite ` 
number of such models in existence and there are many more ways — 
in which the data may occur. If E always mapped psychological 
elements of his universe into the number “43” one model would 
suffice . . . a trivial one indeed. More commonly E maps elements 
into either integers or rational numbers, and the resulting empirical 
density distributions look “something like” the normal curve 
model, ; 

This paper is concerned with the situation in which Æ admin- 
isters different treatments to independent random samples drawn _ 
from the same population and wishes to test the hypothesis: 
vx1 = px2 ` * * = pxx. This implies that Z’s original interest in 
the investigation is formulated in terms of means of a chosen meas- _ 
ure, X, (whether X be the original behavioral measure or some 
transformation of it), and that the alternative hypotheses of in- 
terest also are expressed in terms of the means on this same 
measure. This hypothesis may be tested by obtaining the mean 
square between groups divided by the mean square within groups 
(ms»/ms) from a simple analysis of variance, and tables of the 
F distribution? | 

When the empirical data or a priori analysis suggest a deviation 
from the normal curve model used in the mathematical derivation 
of this statistic, E’s have often chosen to apply a “normalizing 


the words “F test” in this paper may thus be considered ag including this par- 
ticular ¢ test. The usage does not cover repeated measures designs in analysis 
of variance, or the dependent Measures t. 
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transformation” rather than carry out the F test on the non- 
normal X’s. In doing so, they are using a double mapping procedure 
which takes psychological elements to numbers on which the hy- 
pothesis is made (such as equality of means) and then takes the 
numbers into some function (such as logarithms) which is “closer 
to normality.” In this double mapping procedure, the statistic will 
be derived from the second distribution, but the inference is made 
on the first distribution. This procedure might be defended if it 
displayed superiority to the F test on the original non-normal data 
on two criteria: (a) more exact control of the risk of a Type I error 
(rejecting a true null hypothesis). (b) Less risk of a Type II error 
(retaining a false null hypothesis), i.e, a higher power for a 
given degree of violation of the null hypothesis and a given level of 
significance. This study covers both criteria by contrasting the 
power curves of the F test as applied to the original non-normal 
data, and as applied to the “normalized data.” 


Curvilinear Transformations 


The application of a non-linear function, Y = f(X), will alter 
the variance to a degree relative to the mean of the population. The 
desired result of equal variances and normality is obtainable 
theoretically under only two conditions (Lindquist, 1953): (1) 
When the null hypothesis is true and all treatment populations 
have the same form and variance, and (2) When the null hypothe- 
sis is false and all treatment populations fit the same distribution 
function such that the variances can be shown to be a function of 
the means, oj? = f (p). Given one of these situations, a continuous 
measure, and an exact knowledge of the distribution function of the 
populations, a mathematician can then determine the transforma- 
tion that will reduce the data to a normal distribution. In rare 
cases this transformation will be of a simple form and perhaps 
even one of the commonly-used procedures (Y = log X, VX, 1/X); 
but since these are very specific instances, the expected transforma- 
tion usually will be some complex function. A practical problem 
then is that Æ is usually unable to specify exactly the population 
condition, and, with the lack of information about the population, 
the proper transformation is usually indeterminate. 

Considering the second Lindquist condition of correlated means 
and variances, a rough correction can be found if E is not interested 
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in the form of the transformed variables, Curtiss (1943) and Bart- 
lett (1947) have given several mathematical rules which can be 
followed under these circumstances. In this case, if the treatment 
populations are distributed with different means and different 
forms (skewness and kurtosis) there is also some chance that the 
resulting transformed variables will be normal. However, if they 
are of homogeneous form, the different levels of the scores will be 
affected differently by the transformation, and the result will be 
heterogeneous forms on the transformed variate, only one of which 
might be normal. The converse is true in the other Lindquist con- 
dition where the null hypothesis is true. Here, if the forms of raw 
data are heterogeneous and there is no treatment effect, they will 
transform to heterogeneous distributions. } 

Once the transformation is made and the data analyzed, a fur- 
ther theoretical problem confronts Æ in the interpretation of th 
statistics obtained on the transformed scale. Since the statement 
that xı = px, is not equivalent to the statement that man = 
sxx) in general, it is not logically correct to transfer conclusions — 
from the statistical analysis of transformed measures back to the - 
original hypothesis Æ started with. An example where the first 
statement is true and the second is not is given in Table 1. 


TABLE 1 
Relationships between Hypotheses for Two Populations of Data 
Population log Transformations to Base 2 
ASRS 2: ere 
xX X: log Xı log X: 
ua ee ee 
2 8 1 3 À 
2 8 1 3 
2 8 1 3 
2 8 1 3 
2 8 1 3 
32 2 5 1 


EX = 42 EX: = 42 © log X, = 10 X log X: = 16 
ux, =7 ux, =7 Miosx, = 1.67 Hiosx, = 2.67 
bx, = ux, =T Hiosx, ¥ Miogx, 

If E randomly and independently assigns subjects from the same 
parent population to his k treatments, and if the k treatment effects 
are additive and identical, then bx = x2 = ` + + = py, and in 
addition, for any one-to-one transformation it also follows that 
Pri) = Prz) = * * * = uyr). This result occurs because the X 
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populations are identical to each other, and, similarly, the trans- 
formed populations are identical to each other. However, if treat- 
ment effects are not identical, or are not additive (that is, the effects 
of the treatments differ from subject to subject, certainly a com- 
mon state in psychology), then the X distributions will no longer 
be identical. If this is the case, then the hypothesis of equal means 
on X may be true, while the hypothesis of equal means on f(X) is 
false, and vice versa. Rejecting the hypothesis of equal treatment 
means on f(X) may occur because the treatment populations on X 
differ in variance, or in skewness, or in kurtosis, even though the 
population means on X are equal. Thus rejecting the hypothesis 
that ayxi) = pycx2) = * * * = prs) implies some difference in 
the treatment effects, but does not clearly imply the rejection of 
bx. = px2 = ' * * = px. 

The E who is thus interested in the hypothesis of equal means 
on X, but who transforms his measure merely due to form consider- 
ations, thus ends up testing a different hypothesis than originally 
intended, and is typically not logically justified in extending his 
conclusions back to the original measure of interest. The material 
on transformations in many common statistical texts used by 
psychologists is relatively meager (Edwards, 1960; Guilford, 1956; 
Ray, 1962; Walker and Lev, 1953), and few cautions against this 
practice are found. 


Method 


The empirical sampling study which follows contrasts the power 
curves of the hypothesis wx1 = x2 = ps for various violations of 
the normality assumption, when tested by (a) the usual ms,/mswp 
on the X data or by (b) the ms,/ms, ratio on the f(X) data, 
where f(X) is the appropriate transformation that will “normalize” 
all of the k populations when the null hypothesis is true. Since our 
hypothesis is about parameters defined on the X scale, the points 
of the power curve are defined in terrhs of deviations of the means 
in units on the X scale, This is representative of the situation where 
the treatment effects, if any, are strictly additive on X, so that all 
treatment population variances will be homogeneous and equal to 
the variance of the parent population. In short, we are concerned 
with the situation where all the assumptions of the fixed treatments 
analysis of variance model are met except the assumption of con- 
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tinuous measures and the assumption that the common population 
form is normal. The discrete X measures are assumed to be evenly 
spaced. To decrease the possibility of the Central Limit Theorem 
obliterating any effects, the sample size was purposely kept small; 
n = three or six. 

The procedures used were similar to Boneau’s (1962), except that 
a constructed population of 2560 measures was stored in 2560 mem- 
ory locations of an LGP-30 computer, A random selection procedure 
was then used to draw memory locations and treat the value stored 
in that location as X. This procedure was based on a residue-class 
pseudo-random number generator with a period equal to the square 
of 5881. Care was taken to avoid starting the sampling at the same 
point in the generation Sequence during the course of collecting the 
data. The numbers generated were reduced to 12-bit sequences, 
which were interpreted as random addresses of the X values to be 
selected.’ The procedure is the equivalent of random sampling from 
a population of an infinite number of cases but with the exact char- 
acteristics (discrete measures and finite range) of the relative fre- 
quency distribution of the 2560 measures, 

To each of the n measures in a given treatment group, the speci- 
fied treatment effect (ti, ta, or ts) was added, and the MS8y/M8y ratio 
was computed on the resulting measures. This Process was re- 
peated until one thousand ratios were computed. The computer 
output was a frequency distribution of mean square ratios accom- 
panied by an exact count of the ratios exceeding critical F yalues 
in common usage: Fios, Foo, Fox, Fos for k — 1, k(n — 1) degrees 
of freedom. 


For simplicity, the treatment effects were Specified as t, = 0, 
ta = +a, and ts = +2a; thus, a is the distance between the popu- 
lation means in units on the X scale. By varying the arbitrary con- 
stant, a, it is Possible to study any desired points on the power 
curve. The degree of Violation of the null hypothesis is convention- 
ally expressed by the value $, defined as 


i fp SG FG boy 
= Sn re a 


ox 
where ¢ is the average of the ts. 
panne ele 


$ After this study was completed, it was found that the method of extracting 
random addresses led to numbers not uniformly distributed across the 64 sec- 


so 


za 


a 
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Given the value of ¢ and treatment effects specified in a linear 
fashion as above, it is possible to solve for the needed values of a 


by the equation: 
12 
a= NAE 


Thus, when ¢ is zero, a is zero, and the parameter condition 
px + 0 = px + a= px + 2a is that of a true null hypothesis. The 
observed proportion of “significant” F’s obtained when ¢ = 0 is 
thus an estimate of the actual risk of a Type I error. When ¢ is 
greater than 0, the null hypothesis is false, and the larger the ob- 
served proportion of “significant” F’s, the lower the risk of a Type 
II error, Five values of ¢ were studied, 0, .5, 1.0, 1.5, and 2.5. The 
increasing values of ¢ are proportionate to the increasing distance 
between the population means (a). 

Six parent populations of varying forms were used to determine 
the power curves of the F test under various violations of normality. 
One population was a discrete approximation to the normal curve, 
and primarily served to check the procedures used and any effects 
due to discreteness. Symmetrical rectangular and leptokurtic popu- 
lations were used to determine the effects of kurtosis, and three 
unimodal populations varying in skewness (and accompanying lep- 
tokurtosis) were used. These three skewed populations were con- 
structed in such a manner that, when the treatment effect was 
zero, they could be transformed to near normality by three com- 
monly used procedures: log, square root, and reciprocal. These dis- 
tributions are labeled as slight skew (V -N, meaning that the 
square root transformation would “normalize” the distribution), 
moderate skew (log-N), and extreme skew (reciprocal-N). It 
was thus possible to make use of transformations in a more satis- 
factory manner than the typical Æ, in that the transformation used 
was definitely appropriate for the common population shape when 
the null hypothesis was true. Of course, when the null hypothesis is 
false, the transformation was an appropriate “normalizer” only for 
the first treatment population, but this is a hazard implicit in the 
use of transformations in this situation. Table 2 shows the de- 
tors or word spaces of the memory tracks, When this was corrected, the new 
power functions constructed for the normal and leptokurtic populations were 


practically identical to the original. It was concluded that the effect on this 
study due to the biased procedure was negligible. 
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scriptive parameters of the six parent populations, with the param- 
eters of the “transformed populations” listed beneath their appro- 
priate skewed populations. The means and variances of these 
distributions are irrelevant since they have nothing to do with the 
assumption about form except in determining the effects of the trans- 
formations, and the definition of assesses differences in variance 
to establish a similar degree of violation of the null hypothesis for 
all populations. To study the effects of analyzing transformations 
of the data on the probability of rejecting a null hypothesis based 
on the raw data (X), the treatment effects were added prior to the 
transformation. Thus, these computations were carried out on 
VOSS CO Bye 

Skewness is reflected in the parameter, 81. A symmetrical dis- 
tribution will yield zero values of Bı, while positive values indicate 
a concentration of scores to the left of the mean and/or a higher 
range of scores above the mode than below it (positively skewed) ; 
negative values indicate the opposite condition. The kurtosis pa- 
rameter (82) is a measure of degree of concentration of scores in 
the distribution. In the normal curve, B2 = 3.0. Values above this 
are characteristic of “leptokurtic” distributions—ones highly con- 
centrated at a single mode and having long tails. Values of B2: <3 
indicate “platykurtic” distributions—ones approaching uniform or 
multimodel shapes. The mathematical specifications of @, and Bo 
follow Pearson (1931). Although it is very common to obtain 
leptokurtic or platykurtic distributions with no skewness, most 
unimodal skewed distributions also show leptokurtosis, Thus on 
empirical data, skewness is typically accompanied by leptokurtosis. 

It may be noted in Table 2 that the discrete normal approxima- 
tion and those distributions transformed to normality show a kur- 
tosis index slightly below the expected 3.00. This is apparently due 
to the finite size of the populations, which necessitated the exclusion 
of extreme values with a probability of less than 1/2560, and thus 
truncated the ends of the distributions. The discrepancy between 
the continuous distributions of mathematical theory and the dis- 
crete distributions of recorded data is 


Tpu brought out another way 
when considering transformations. Figure 1 shows the discrete ex- 


tremely skewed (reciprocal-normal) distribution, contrasted to the 
“normalized” transformed distribution. The “normalizing” of such 
discrete distributions can only be accomplished by spreading out 


i 


T 


Pi 
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points at one portion of a scale and contracting points at other 
portions. In this example, there are only eight score values covering 
the distance to the left of the mean of the “normalized” distribu- 
tion, while 52 values are found to the right of the mean. The per- 
centile rank of the mode of this “normal” distribution is 36.3. 


10 15 20 25 30 35 40 45 50 55 60 65 


x 
055 0955 .0855 .0755 .0655 0555 „0455 0355 0255 .0155 0055 
a 


Figure 1. Frequency polygons for the extremely skewed (Rec-N) popula- 
tion and its corresponding “normalized” population, the reciprocal-transform. 


It should be noted that f(ux) = ur Only when the X population 
meets very restricted conditions specific to the transformation 
involved. For example, for the case Y = log X: ny = log ux only if 
u} = T"X, the product of all the X values. Thus, in general, E can 
not take the complementary transformation of the mean of the 
transformed variable as equivalent to ux. For the distribution in 
Figure 1, the reciprocal of the mean of the 1/X distribution is 18.5, 
about 1.4 points below the population mean of the X distribution. 


Results 


All ms,/msy distribution obtained when the null was true were 
tested for goodness of fit to the theoretical F curve with appropriate 
degrees of freedom. Only two of the 18 tests were significant at the 


aa EXTREMELY SKEWED (REC-N) 
2———- RECIPROCAL TRANSFORM 
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.05 level, those for samples of size n = three from the leptokurtic 
population and the rectangular population (Chi Squares of 38.91 
and 65.08 respectively, both with df = 24). However, since these 
tests are not sensitive to deviations in the tails of the sampling dis- 
tributions, the area of greatest interest, the results are best ex- 
pressed in terms of the proportions of the sampling distributions 
that exceeded the theoretical critical values of F, or the empirical 
estimate of the power of the test under the given condition. Tables 
2 and 3 show the empirical power estimates using the .05 signifi- 
cance level with an n of three and n of six respectively.* Theoretical 
power values were approximated for the non-central F distribution 
using a procedure developed by Severo and Zelen (1960). Critical 
values for the significance of deviations in power were computed 
for the .05 level and are listed in the bottom row of each table. De- 
Viations in empirical estimates from the theoretical power are listed 
and are marked with an asterisk when significant. 

It is evident from Tables 2 and 3 that the normal distribution 
sampled in this study resulted in power values not significantly 
different from the theoretical. This indicates that the procedures 
used in this study were sufficiently valid to make consistent devi- 
ations meaningful indications of the effects of non-normality. It 
can also be seen that the slightly skewed and moderately skewed 
Populations gave very little deviation from theoretical power for 
both sample sizes, with a general trend to increase the power in the 
Tange of moderate degrees of falsity of the null. Transforma- 
tions of these two distributions show slightly greater effect, usually 
in the same direction as that of the original distribution, 

The greatest deviation from theoretical power curves resulted 
from sampling the two highly leptokurtic populations, the extremely 
skewed, where 22 = 9.54, and the symmetrical leptokurtic, where 
Ba = 9.16. All four curves involving these populations show the 
test to be slightly conservative when the null is true ($ = 0), but 
to be appreciably higher in power than expected when in the re- 


1%, and 5% significance levels; 
number of ratios exceeding the 
ese tables less stable than those 
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gion of moderate departures from the null (6 = 1.0, 1.5). The 
power curves for samples of n = 3 from these populations are con- 
trasted with the theoretical curve, and the curve for the platy- 
kurtic rectangular population in Figure 2. The rectangular, or platy- 
kurtic, distribution (f2 = 1.80) shows an effect opposite to that 
of the leptokurtie cases, although it lies closer to the theoretical 
curve. The reciprocal transformation of the extremely skewed pop- 
ulation (dashed line in Figure 2) in general surpassed the power of 
the test on the raw data, except at ¢ = 2.5, where a loss is quite 
apparent. The increase in risk of a Type I error when ¢ = 0 is gen- 
erally true for transformations (causing an average increase of 
006 over the risk on the raw data), although none of these differ- 
ences is significant. Only in the case of the large sample size and 
extremely skewed distribution did a transformation improve the 
accuracy of the test when the null was true. 


Discussion 


This empirical sampling study was designed to help the experi- 
menter interested in the equality of treatment effects on the X 
scale to choose between (1) conducting the usual msp/m8w test on 
data known to be non-normal, or (2) transform the data by some 
procedure to a shape closer to normality and test the means of the 
transformed variate. Although the results obtained are specific to 
the particular populations used and undoubtedly contain some sam- 
pling variability, the results show consistent trends. 

For the slightly skewed and moderately skewed distributions, the 
deviations from the theoretical power curve were 80 small that the 
extra trouble of using transformations would seem sufficient cause 
to abandon them. The fact that the use of the appropriate trans- 
formation increased the deviation from the theoretical power more 
often than it decreased it raises further questions about this prac- 
tice. In an actual experiment, E typically has no adequate basis for 
choosing the “most appropriate” transformation. Furthermore, 
since this practice involves either the use of the unlikely assump- 
tion that the hypothesis is the same on X as on the transformed 
measures or else requires an embarrassing change of hypotheses at 
the end of the investigation, the authors see no justification for 
continuing this practice on such distributions. 


PROPORTION OF MS,/ms EXCEEDING Fos 
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The only substantial deviation from the normal power curve was 
for the two populations that were highly leptokurtic—the extremely 
skewed population and the symmetrical leptokurtic population. 
The power curves for tests on these populations are consistent with 
the theoretical results of Srivastava (1959) for continuous dis- 
tributions: the risk of a Type I error is slightly less than specified, 
with the negative deviation at ¢ = 0 changing to positive devia- 
tions at points of intermediate power, and with a second crossing of 
the theoretical power curve occurring at high power points when the 
deviation again becomes negative (as at øg = 2.5 when n = 6). 
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k pate 2, Empirical power curves for certain nonnormal populations, k = 3, 
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Thus, the trends found by Srivastava may be extended to discrete 
populations with considerably higher £, values than could be in- 
vestigated by his methods, Oddly enough, these results suggest that 
leptokurtic data distributions might improve the F test. The con- 
servative test when the null is true would be to err on the safe side, 
and the increased power through moderate degrees of falsity of the 
null would give the researcher greater power where it is typically 
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most needed (Cohen, 1962). The fact that the empirical power is 
slightly lower for very high power points is a slight disadvantage 
but one that E might be willing to exchange for relatively greater 
increases in power at intermediate levels. No simple common trans- 
formation could “normalize” a symmetrical leptokurtic distribu- 
tion; the “normalizing” reciprocal transformation increased the 
amount of discrepancy from the theoretical curve in seven of ten 
comparisons. 

The problem of platykurtosis may be judged more serious by 
many Es, since this condition tends to have effects opposite to 
those above and thus increases the risk of a Type I error above the 
specified significance level. However, the authors would consider the 
deviations shown for the rectangular population on samples of six 
cases to be quite minor, and Srivastava’s work (1959) indicates 
that increasing n decreases the kurtosis effects on the power curve, 
The authors feel the rectangular population (B2 = 1.8) is quite ex- 
treme for psychological data and would feel reasonably safe using 
the conventional F test with equal ms of ten or more. If bimodal 
populations of extreme platykurtosis were encountered (minimum 
possible 82 is 1.0), larger n’s would be advisable. 

The results of this study and of Srivastava’s theoretical work 
contradict the Norton (1952) results showing that leptokurtosis 
increased the risk of a Type I error. The difference may be due to 
the fact that Norton sampled exhaustively, without replacement, 
from a finite population, while the present study sampled randomly, 
with replacement, from a finite population. 

Tt should be emphasized that the present study has dealt only 
with use of transformations to correct for violations from normal- 
ity. There are many other usages of transformations that have not 
been, and are not being, criticized. The use of a transformation is 
essentially selecting 4 scale of measurement. Mueller (1949) 
pointed out that some theories may specify the use of a transforma- 
tion by, say, using log X as the scale of measurement in the theory. 
To test the theory then, measures should be transformed as re- 
quired. Other times, a transformation may be used to simplify the 
expression of a relationship that requires a more complex equation 
when expressed in X. Occasionally, a curvilinear relation between 
W and X may be linear between W and log X; if the homo- 
scedasticity also improves, then a considerable simplification in 


+ 
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description is possible. These are pertinent reasons for selection 
of a scale of measurement; what is being said is that often the fact 
of non-normality alone is NOT a sufficient reason for changing 
from an a priori acceptable scale of measurement to some other 
arbitrary scale. In the analysis of variance of independent groups, 
in particular, the disadvantages of this procedure seem to out- 
weigh the advantages, 

In addition to normality, transformations have also been recom- 
mended to achieve homogeneous variances, or additivity of treat- 
ment effects. It should be noted that these three goals will not nec- 
essarily be met by the same transformation, although it is possible 
for this to occur. Box (1954) has shown that homogeneous vari- 
ances are not too important in analyses of variances of independ- 
ent groups with equal ms, although this goal may be more im- 
portant in repeated measures designs and prediction uses involving 
homoscedasticity. Similarly, lack of additivity does not disable a 
simple analysis of variance of independent groups. However, ad- 
ditivity would be quite important in a design that assumed no 
interactions, 

In general, the use of a clearly interpretable scale of measure- 
ment certainly should be the dominant consideration. If the sta- 
tistical characteristics of the scale are not ideal, often an experi- 
mental design that is not greatly impaired by the defect may be 
found. Statements by Tippett (1952, p. 844-345) best summarize 
the argument. “If a transformed variate (Y), having convenient 
statistical properties can be substituted for X in the technical 
arguments from the results and in their applications (italics ours) 
eg is everything to be said for making the transformation. 
md ae, nar saa 
formed on (Y), means and so on ha’ oh ed 
statistical difficulties arise, and th ie aaa 

4 ) waters deepen. Readers are ad- 
vised not to make transformations on statistical grounds alone un- 
less they are good swimmers and have experience of the currents.” 
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“FAKING GOOD” WELL? 
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Tum apparently widespread tendency of subjects to put their best 
foot forward when responding to items from personality and ad- 
justment inventories has remained largely unanalyzed in the wake 
of recent efforts to detect, discourage or bemoan such behavior 
(Hanley, 1957; Voas, 1958; Edwards, 1957). Variations in the ap- 
parent strength of this tendency under different circumstances of 
assessment suggest that the phenomenon is in part a situational 
one. Even within a single assessment context, however, considerable 
individual differences in ‘this tendency may be observed, sug- 
gesting, among other things, a person-situation interaction which 
reflects motivational characteristics of the subjects. Moreover, there 
is some evidence which suggests that when subjects are explicitly 
told to make a good impression, those who do “best” are likely to 
be more psychologically sound than those who do less well at this 
task (Grayson and Olinger, 1957; Canter, 1963). The recent re- 
vival of interest in “role-playing” studies has produced a consid- 
erable amount of data which supports the observation that even 
under instructions designed to induce a uniformly high degree of 
good impression motivation, subjects will vary extensively in their 
performance (Wiggins, 1959; Walker, 1961; 1962). The present 
study was designed to test a hypothesis concerning one possible 
source of these observed individual differences. 

Whether “psyching out” a potential employer or cooperating 
with the explicit instructions of a role-playing experiment, making a 

1 This investigation was supported by Public Health Service Research Grant 
No. MH 07042-01 from the National Institute of Mental Health. 
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good impression via a personality inventory would seem to involve 
an ability or “shrewdness” component as well as a motivational 
one. Given the motivation to fake, one must know the standards of 
the test scorer with respect to what will be judged favorably. The 
social desirability rating procedures developed by Edwards (1953) 
provide an excellent paradigm for discussing this process. In Ed- 
wards’ terms, the subject must attempt to estimate the social de- 
sirability value of an inventory item and respond accordingly 
(Edwards, 1957). A correlated, and perhaps indistinguishable, 
hypothesis would be that the subject attempts to estimate the pro- 
portion of people who would admit to the item (communality value) 
and responds accordingly (Wiggins, 1962). Whether the subject is 
estimating “desirability” or “popularity,” he is nonetheless esti- 
mating and he may do so accurately or inaccurately. The present 
study tests the hypothesis that accuracy of desirability and com- 
munality estimations are positively related to the degree of suc- 


cessful faking a subject attains under explicit instructions to make 
a good impression. 


Method 
Subjects » : ° 


The subjects were 239 male undergraduates who were assigned to 
this study early in the semester as part of the course requirement in 
introductory psychology at the University of Illinois. Subjects 
were randomly assigned to one of six experimental conditions which 
each required approximately one hour of their participation. 


Experimental Design 


The experimental conditions differed both in the tasks required 
of the subjects and in the order in which these tasks were pre- 
sented. Two sets of stimuli (items) were presented to all but one of 
the six groups thus formed. One set of stimuli consisted of 50 
items and will be referred to as the rating form. The other set of 
stimuli consisted of 212 items and will be designated the inven- 
tory. Three sets of instructions were employed with the rating form: 
(a) estimate communality, (b) estimate social desirability or (c) 
rate social desirability. Two sets of instructions accompanied the 
inventory: (a) self-report or (b) fake good. 
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Table 1 illustrates the order of stimulus presentation and kinds 
of instruction which define the six separate subject groups. The 
Communality Estimation group estimated communality on the rat- 
ing form and faked good on the inventory. Half of the subjects in 


TABLE 1 
Experimental Design (N = 239) 


ORDER OF 
GROUP DESIGNATION N PROCEDURES 
CE; 21 Est. Comm. Fake 
Communality Estimation 
CE: 25 Fake Est. Comm. 
DE: 23 Est. SD Fake 
Desirability Estimation 
DE; 25 Fake Est. SD 
Desirability Rating DR 50 Rate SD Self-Report 
Self-Report SR 95 Self-Report only 


this group estimated communality first (CE,) and the other half 
faked good first (CE2). The Desirability Estimation group estimated 
desirability on the rating form and faked good on the inventory. 
The order of stimulus presentation was also counterbalanced within 
this group. Subjects in the Desirability Rating group first estimated 
desirability and then gave their self-report to the inventory. The 
Self-Report group simply gave their self-report to the inventory. 


Rating Form sa 


The 50 items employed for the Tating form did not overlap with 
those appearing in the inventory. The rating form items were se- 
lected in such a way as to maximize covariation in rated social de- 
sirability value and endorsement percentage. There were three 
versions of the rating form which were alike in that the same items 
were to be judged on an 11-point seale. The versions differed in 
instructions and in the descriptive anchors accompanying the 11- 
point scale. The three rating tasks were as follows: 

Communality Estimation (CE). Subjects who received this ver- 
sion of the rating form were asked to give their best estimate of 
the proportion (per cent) of men in their class who would answer 
True to each of the 50 statements. They did so on an 11-point rat- 
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ing scale of percentages: 0%, 10%, 20% .. . 100%. These subjects 
were, in effect, being asked to estimate the endorsement percentages 
which would occur in a group such as the SR group. 

Desirability Estimation (DE). Subjects who received this ver- 
sion of the rating form were asked to estimate which of a number of 
rating categories would best represent the opinion of men in their 
class with respect to the desirability of a True answer to each of 
the 50 statements. Ten rating categories were provided in the form 
of an 11-point scale: 0 (very extremely undesirable), 1 (extremely 
undesirable), 2 (strongly undesirable) ...10 (very extremely de- 
sirable), 

Desirability Rating (DR). Subjects who received this version of 
the rating form were ‘asked to give their opinion of the desirabil- 
ity of a man in their class answering True to each of the 50 state- 
ments. The same 11-point rating scale of desirability employed by 
the Desirability Estimation (DE) group was provided. In a sense 
which will be amplified later, the subjects in the DE group were 
trying to estimate the pooled or averaged values obtained from 
the DR group. 


Inventory á 


The 212 item inventory was comprised of items from MMPI 
scales which had previously been shown to be sensitive to role- 
playing instructions. Rosen (1956a and b), Walker (1961; 1962) 
and Wiggins (1959) have presented data on MMPI clinical and 
stylistic scales which exhibit significant shifts in scale means under 
role-playing instructions, The first two of these studies obtained 
repeated measurements (standard-fake) on the same subjects 
while the third study contrasted the scale Scores of independent 
groups (standard and fake). Scales were selected for the present in- 
ventory from among those which had exhibited significant mean 
score shifts under role-playing instructions in at least two of the 
three independent studies (Rosen, 1956a and b; Walker, 1961; 
1962; Wiggins, 1959), MMPI clinical scales which exhibited a 
significant decrease in these studies were Pt (McKinley and Hath- 
away, 1942), A (Welsh, 1956), Hy-O and D-0 (Wiener, 1948). 
antly increased were Hy-S and 
) stic scales which increased sig- 
nificantly under role-playing instructions were Sd (Wiggins, 1959), 
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Mp (Cofer, Chance and Judson, 1949) and L (Meehl and Hath- 
away, 1946). 

The 212 item pool which included all items from the preceding 
nine scales was presented in randomized order in the standard in- 
ventory format with two response options (True and False). Two 
sets of instructions were employed with the True-False inventory: 

Self-Report (SR). The self-report instructions were adapted 
from those utilized in the standard administration of the MMPI 
(Hathaway and McKinley, 1951). Subjects were instructed to mark 
a statement True if it applied to themselves and False if it did not. 

Faking Instructions. The role-playing instructions employed in 
the present study attempted to maximize the amount of social de- 
sirability responding in accord with suggestions along this line 
made by Walker (1962). The instructions read, in part: 


“We want to see how well you can do in answering these state- 
ments so as to create the most favorable impression you can. Pre- 
tend that your answers to these statements will be read out loud 
in class. Assume also that you are especially. concerned with mak- 
ing a good impression on your classmates. Now answer each 
statement in such a way as to create the best impression on 
others in the class .. . Remember, you fre NOT being asked 
whether the statements are True or False as applied to yourself. 
Rather, you are being asked to answer each statement in such a 
way as to create the best possible impression on others in your 


class.” sa 
Accuracy Measures * 

Communality Estimation. As previously mentioned, 95 subjects 
in the group designated SR took an inventory under self-report 
instructions only. This inventory was an augmented version of the 
212 item inventory which included, in addition, the 50 items from 
the rating form; here in true-false format. Item endorsement per- 
centages were computed for these 50 items in the group of 95 
standardly instructed subjects. These endorsement percentages were 
rounded to the nearest tenth (0%, 10%, 20% ... 100%) to serve as 
the criterion indices of accuracy of communality estimation. An ab- 
solute accuracy score (AA) was computed for each subject by 
summing the discrepancies between estimated and obtained endorse- 
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ment percentage across the 50 items without regard for the direction i 
of such discrepancies. In addition, a relative accuracy score (RA) _ 
was computed across all 50 items which was based on the algebraic 
differences between estimated and obtained endorsements. 

Desirability Estimation. As already indicated, 50 subjects in the — 
DR group were asked to give their own opinions as to the desirabil- 
ity (11-point scale) of their classmates subscribing to the 50 rating 
form statements. The median desirability values of the subjects in 
this group served as the criterion indices of accuracy of desirability 
estimation for subjects in the DE group. These median desirability 
ratings were rounded to units (0, 1, 2,...10) for purposes of com- 
parison with the individually estimated desirability ratings of the 
DE group. Two indices of desirability estimation accuracy were ~ 
computed in the same manner as those for communality estimation ` 
accuracy. These were: an absolute accuracy score (AA) based on 
summed absolute discrepancies between estimated and obtained A 
desirability values, and a relative accuracy score (RA) based on 
summed algebraic discrepancies. 


Faking Measures 


A total of 94 subjects was ANNE to make a good im- 
pression on the 212 item inventory in a design counterbalanced for 
the possible order effects of the additional tasks of desirability and 
communality estimation. The somewhat knotty problem of opera- 
tionalizing faking “success” was met by resort to scale scores on 
indices which had been previously demonstrated to be sensitive to 
role-playing instructions in both test-retest and contrasted group 
studies. High scores on the individual dissimulation scales Sd, Mp 
and L were considered evidence of faking success as was the sum of 
such “fake” scales: Sd + Mp + L. Low scores on the pathology 
scales Pt, A and Hy-O were taken as indicative of successful faking 
as was their sum: Pt + A + Hy-O. A gross index of faking success 
was also based on the difference between the preceding sum scores. 
Additional clinica] scales were Hy-S and D-S which were expected 


` to increase and D-O which was expected to decrease with success- 
ful faking. i 


In deference to the lack of un: 
“best” index of dissimulation ( 
ker, 1962) several such indice 


animity of opinion as to the single 
Wiggins, 1959; Hanley, 1961; Wal- 
S Were scored in an admittedly ex- 
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ploratory fashion. It should be noted, however, that the scale scores 
from the group of 95 subjects who took the inventory under standard 
instructions (SR) provided a baseline against which to evaluate 
the relevance of individual scales to the present study. 


Results 


Relevance of Faking Indices 


The nine indices of faking success which had been selected from 
previous studies were scored separately for the present sample of 
95 standardly instructed men and the 94 men who had received fak- 
ing instructions. Table 2 presents mean scale scores for the two 


TABLE 2 
Mean Scale Scores on Faking Indices in Fake Good and Control Groups 


Group L Mp. Sd Pt A  Hy-O Hy-S D-0 


11.20 13.87 15.73 ẹ 9.01 9.10 7.35 5.39 5.78 
p .001 .001 .001 .001 .001 .001 .001 001 
— 


groups along with the ¢ values for the differences between the groups. 
The hypothesis that the obtained mean scores for the faking and 
control groups could have arisen from the same population was re- 
jected at less than the .001 level for all nine faking indices. This ex- 
pected result is necessary but not, sufficient to justify the employ- 
ment of these measures as indices of faking success in the present 
study. 


Other Effects * 


The counterbalanced nature of the present design permitted an 
assessment of the influence of the tasks of communality and desira- 
bility estimation upon subsequent faking success as measured by 
the several dissimulation indices of the inventory. Mean scale scores 
of subjects who had participated in the estimation procedures prior 
to the faking situation (Groups CE, and DE,) were contrasted 
with mean scale scores of subjects who faked the inventory prior to 
the estimation procedures (CEs and DEz). 


Fake Good Mean 7.46 2142 2445 646 5.62 2.28 17.88 5.68 11.65 

(N = 94) o 3.51 545 6.09 6.05 5.46 3.96 3.51 4.48 

Control Mean 2.87 11.85 12.81 15.02 14.54 6.25 14.97 9.44 10.19 

(N = 95) o 184 3.95 3.73 6.90 7.76 3.38 3.95 4.45 
t 


‘001 


336 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


For subjects in the Communality Estimation groups, the mean 
faking scale scores were highly similar for the group which faked 
prior to estimation and the group which faked after estimation. None 
of the nine mean scale score differences approached significance. 

In contrast, three of the nine dissimulation scales were found to 
have different means under different orders of administration in the 
Desirability Estimation groups. Subjects who had desirability esti- 
mation experience prior to faking exhibited higher mean dissimula- 
tion scale scores on Sd (t = 4.22, p < 001), Mp (t = 2.87, p < 01) 
and L (t = 1.80, p < .10). None of the remaining dissimulation 
scale score differences approached significance. 

Scales Sd, Mp and L have similar and interesting empirical prop- 
erties which may be related to the present order effects, In role- 
playing studies they exhibit not only large mean score differences 
between faking and control groups, but high indices of discrimi- 
native efficiency as well (Wiggins, 1959). Further, they define a dis- 
tinct factor of the MMPI which is generally agreed to be a dissimu- 
lation factor (Edwards, Diers and Walker, 1962; Wiggins, 1964; 
Liberty, Lunneborg and Atkinson, 1964). 

A common property of Sd, Mp, and L as well as other dissimula- 
tion scales belonging to this factorial grouping (such as the Mar- 
lowe-Crowne (1960) Social Desirability Scale) is that item desira- 
bility values tend to exceed empirical endorsement frequencies. 
This fact has been variously emphasized as reflecting the “obvious- 
ness” of such items (Hanley, 1961), the dishonesty of subjects 
answering such items (Edwards et al, 1962) and the psychometric 
properties of items which can in fact change under special instruc- 
tions (Messick and Jackson, 1961; Wiggins, 1962). The present re- 
sults indicate that specific practice in social desirability estimation 
will enable subjects to achieve higher scores on faking scales subse- 
quently administered. Such results do not allow a choice among the 
alternate emphases given to these dissimulation scales. 


Accuracy and Faking 


Social Desirability Estimation. The substantial and unanticipated 
differences in dissimulation scale scores between the counterbalanced 
Desirability Estimation groups required separate analyses of the re- 
lation between accuracy and faking in these two groups. Correla- 
tions between the two accuracy measures and the several indices of 
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TABLE 3 


Correlations between Two Measures of Accuracy of Desirability Estimation and 
MMPI Indices of Faking Success for Subjects who Estimated Prior 
to Faking (DE;) and Subjects who Faked Prior to Estimating (DE?) 


L Mp Sd 2F Pt A Hy0O =P 2F-=P Hy-S D-O D-S 
Group DE; AA —04 —10 —25 —16 47* 48* 29 46" —36 —48* 22 —25 
(N = 23) RA —35 —39 —30 —38 41 36 12 34 —44* -23 19 —18 
Group DE, AA —14 —15 —18 -17 15 17 -14 10 —16 07 —07 -20 
(N =25) RA —21 —18 —11 —17 04 19 -O1 11 -—16 -—10 -05 —10 


*p < 05 


faking success are presented in Table 3 for the group which esti- 
mated desirability prior to faking (DE,) and the group which 
estimated desirability after faking (DE2). In the group which esti- 
mated prior to faking, relative accuracy (RA) is related consist- 
ently to the dissimulation scales (L, Mp, Sd)? This suggests that 
tendencies to overestimate the social desirability of items are re- 
flected in increased dissimulation scale scores under faking instruc- 
tions. This trend is not statistically reliable in this small group of 
subjects, however. 

In contrast with the findings for dissimulation scales, several of 
the pathology indices are consistently and reliably related to accu- 
racy of social desirability estimation in Group DE. Absolute ac- 
curacy scores (AA) are positively and reliably correlated with re- 
ductions in Pt, A and Sum Pathology and increases in Hy-subtle 
under faking instructions. The same trend is apparent for the relative 
accuracy measure. When Ss are asked to estimate the social"desira- 
bility of a group of items prior to faking good on a different inven- 
tory, their accuracy of such estimation is related to their subsequent 
success in obtaining low pathology scores (A, Pt) and high scores 
on a subtle hysteria scale (Hy-S). 

A quite different picture emerges when the accuracy versus faking 
success correlation matrix is examined for the group which faked 
prior to estimating desirability (DE). Although some of the same 
patterns are present there are many reversals of trend and none of 


2In interpreting the signs of these correlations it should be noted that the 
accuracy scores are actually “inaccuracy scores” so that negative relations are 


expected with dissimulation scales and positive relations with pathology scales. 
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the correlations attains statistical significance. When Ss are asked 
to estimate the social desirability of a group of items after faking 
good on a different inventory, there is no consistent relationship be= 
tween their accuracy of estimation and their prior success in obtain- 
ing low pathology scores or high dissimulation indices. 

Communality Estimation. Since no order effect was detectable in 
the communality estimation group it was possible to base the cor- 
relations between accuracy and faking success on the combined” 
group of 46 Ss. The accuracy indices were slightly and often incon- 4 
sistently related to the dissimulation indices. Relations between 
accuracy and pathology score reduction were consistent in sign but 
rather clearly not significantly different from zero. Only the relation ` 
between communality overestimation (RA) and increase in subtle 
hysteria (Hy-S) was significantly different from zero (r = .36). Th 
accuracy with which Ss estimate communality is not obviously 
related to their success in reducing pathology scores or increasing — 
dissimulation indices when faking a separate inventory. > 

Discussion 

The present findings do not permit a simple formulation of the 
relationships between accuracy of popularity or desirability estima- 
tion and role-playing behavior. The unanticipated order effects in 
the desirability estimation group place a severe restriction on evalu- 
ation of the main hypothesis, yet suggest other post hoc lines of — 
reasoning which are perhaps of greater interest. j 

The findings with respect to the communality variable appear to be — 
the most straightforward. The lack of order effects with this vari- 
able suggest that it is not, in itself, an influential contributor to — 
the “set” or strategy which Ss adopt in role-playing tasks. The — 
meager and inconsistent relationships between indices of communal- _ 
ity estimation accuracy and faking success are not supportive of the — 
original hypothesis that this type of estimation is an important — 
parameter of individual differences in role-playing situations. Al- 4 
though not directly compared with the desirability estimation pro- 
cedure, the communality variable seems clearly less potent as an 
explanatory construct. A similar lack of explanatory power for 
the communality variable (as an alternative to social desirability) _ 

? 


ea reported in a quite different context (Jackson and Messick, 
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The order effects between the social desirability estimation groups 
suggest that the actual experience of such estimation modifies 
the subsequent “set” or strategy the S adopts in the role-playing 
task, The fact that this order effect manifests itself primarily in one 
factorial grouping of role-playing dissimulation scales (Sd, Mp, 
L) suggests that Ss who adopt this as part of their approach to 
role-playing are, in principle, detectable by this group of scales. 
Once the S adopts this approach to role-playing, the accuracy with 
which he is capable of estimating desirability becomes a determinant 
of the degree of faking success he attains as measured by another 
factorially distinct group of scales (Pt and A). 

A considerable body of evidence exists (Edwards et al., 1962; 
Wiggins, 1964; Liberty et al., 1964) which indicates that Edwards’ 
social desirability scale (SD) belongs to this last mentioned 
factorial grouping (Pt, A) and has similar correlates. It may reason- 
ably be supposed then that individual differences in accuracy 
will be related to differences in scale score on Edwards’ SD scale 
for Ss who adopt this role-playing strategy. Should subsequent ex- 
perimentation substantiate this inference it might shed light on some 
interpretative differences between social desirability measures be- 
longing to the first factorial grouping of the MMPI (SD) and those 
belonging to the third (Sd, Mp). Subjects who are high on Sd 
but low on SD might be Ss who have adopted, at least in part, the 
role-playing strategy which emphasizes desirability estimation, but 
who lack the accuracy of judgement to implement it. Related 
hypotheses readily occur with respect to other combinations of these 
two scales. 
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DIMENSIONS OF GROUP INTERACTION: THE 
COOPERATIVE ANALYSIS OF IDIOSYNCRATIC 
DESCRIPTIONS OF TRAINING GROUPS: 


JOHN L. RINN? 
University of California, Berkeley 


Tum concern in this study is with the investigation of the per- 
ceptual-cognitive parameters underlying the observation, descrip- 
tion, and rating of social behaviors. The study also develops a 
method for measuring the changes over time in descriptions of 
group behaviors. The method remains close to the data in that the 
rating scales are cooperatively developed, and in that ratings are 
performed on “own-statements” rather than on descriptions given 
by other observers. The study involves certain methodological inno- 
vations which have implications for research done within a phenom- 
enological framework. From the viewpoint of graduate education 
in the social sciences, the procedures promise to contribute to such 
educational goals as skill in perceiving the verbal and nonverbal 
behaviors of participants in a group endeayor, skill in formulating 
perceptions of interpersonal behavior in communicable form, and 
skill in constructing research instruments for the investigation of 
perceptual-cognitive data. 


Procedures 


Members of a seminar in group processes at the University of Cali- 
fornia, Berkeley, were organized into two training groups of about 
15 members each and were encouraged to deal with various develop- 
mental tasks which could be solved on the basis of group planning 

1 Supported by funds provided through the Department of Education Re- 


search Committee, University of California, Berkeley. 
2 Now in private practice as a psychologist in Berkeley, California. 
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and group action. The members held secondary school teaching 
credentials and were preparing themselves for positions as counse- 
lors. Their ages ranged from 25 to 35, and each group held approxi- 
mately the same number of men and women. The educational pur- 
poses of this procedure were (1) to provide experiential referents 
for the theoretical constructs which the students would encounter 
in reading the literature on group processes, and (2) to provide 
an opportunity for developing skill in interpersonal observation 
and interpersonal influence in a group setting. 

At the end of each weekly 14-hour meeting, members were asked 
to record their “most significant” observations by writing four de- 
scriptive statements which completed the following sentence stems: 
(1) The group .--, (2) The leader ..., (3) One member..., and 
(4) L.... These statements were collected, collated, and fed back to 
the group at later meetings so that the members could compare 
member interests and so that they could become more sensitive to 
the variations among members in perceptual focusing and com- 
munication style. These feedback materials frequently led to a dis- 
cussion of topics such as the extent to which different statements 
were descriptions of the same event, or how difficult it was to clarify 
the relationship between what a person intends to say, what he 
says, and what others think he says. | 

‘Near the end of the semester, the instructor introduced the sugges- 
tion that these descriptive statements could be used to assess the 
directions of group development, and that the members could ap- 
proach this question as a cooperative research task. One of the 
groups rejected this suggestion, and the other accepted it and went 
to work on it. The balance of this paper, then, refers to the data 
provided by one 15-member training group. 

Upon acceptance of the research task, one meeting was devoted 
to the consideration of categories into which statements could be 
fitted and, particularly, of the dimensions along which it was hy- 
pothesized that changes over time might have taken place. An im- 
portant criterion was that all members felt that classification of 
statements in terms of each category was psychologically feasible. 

The result of this step was a set of seven scales which could be 
used to rate the statements which had been collected over a period 
of eleven meetings. Each of the scales turned out to be bipolar in 
nature, and a simple seven-step rating sheet was devised which of- 
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fered a neutral point if the scale was judged to have no relevance 
for a particular statement. The scales were defined in bipolar terms 
as follows: 


1. personal-impersonal 

2. internal-external 

3. participation-nonparticipation 
4. interest-apathy 

5. sensitivity-insensitivity 

6. dominance-submission 

7. satisfaction-dissatisfaction. 


Prior to the last meeting of the group, each member was given 
his own set of statements to rate, the statements having been sepa- 
rated and coded so that the actual ordering in time was not indi- 
cated. The purpose of this step, of course, was to minimize the effect 
of any preconceptions on the part of members as to which scales 
should show change over time. The effect of memory could not be 
eliminated, but it does not seem reasonable that it could have any 
systematic effect when it is noted that 7 X 4 X 11 separate rat- 
ings were involved for each person. 


Analysis of Data 
Scale Time Trends 


To determine whether or not discernable time trends existed in the 
data, the mean ratings for each session for each type of statement 
were calculated. It should be noted that the averages were taken 
for a single scale (¢.g., personal-impersonal) and for the same meet- 
ing but that the statements which were rated were not identical, 
each person rating his own statements. Thus, although each state- 
ment was a response to a common stimulus stem (e.g., “The group 
...”), the selection of content was idiosyncratic. The appearance of 
time trends, therefore, would be rather strong evidence that 
group phenomena were potent enough to overcome both the inter- 
person variance due to perceptual selectivity and that due to dif- 
ferences in rating-scale response. 

The 28 mean ratings (7 X 4) were first examined “by eye” in 
their graphical form, and smoothed averages were also drawn. Visu- 
ally, it appeared that several trends were in evidence but since 
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the scales showed considerable covariance, factor analytic proce- 
dures were called upon. 


Correlational and Factor Analysis 


Each of the four statement categories was treated as a separate 
social stimulus, and in each case a three-dimensional array of data 
was available consisting of 15 subjects by seven scales by 11 oc- 
casions. This array was first collapsed into a two-dimensional ma- 
trix by taking the mean rating over subjects of each scale on each 
occasion. Intercorrelations of the seven scales were thus based on 
mean ratings of 15 subjects over 11 occasions. The appropriateness 
of the procedure depended on the assumption that there were no 
consistent biases among the subjects concerning how the scales 
were to be applied to the stimulus statements, and this assumption 
seemed to be congruent with the fact that the scales were cooper- 
atively developed by the subject-raters. The correlation matrices 
are presented in Table 1. 

The presence of different patterns among the four correlation 
matrices suggested the possibility that scale interrelationships 
were to some extent a function of the social object that was being 
described. Therefore, the four matrices were subjected to separate 
centroid factor analyses, and an attempt was made to interpret the 
factors. 

The unrotated centroid factor loadings are presented in Table 2. 
The four factor matrices were rotated orthogonally by hand in 
terms of the two criteria of simple structure and comparability 
among the four solutions. Table 3 gives the rotated loadings of all 
factors. Factors were not rotated unless théy contained loadings of 
.50 or more. Thus, only the first two centroid factors were rotated 
except for the “One member” solution. This solution was the least 
satisfactory, although it appears to reflect reasonably well the cor- 
relation matrix from which it was derived. It retained some sizable 
loadings on the third factor which were judged to be specific to the 
particular stimulus object involved. 

The visual record of this operation can be seen in Figure 1 where 
the graphs of Factor I against Factor II are given for the four 
stimulus objects. It can be seen that the rotational criteria were 
reasonably well met, and the similarities in the four solutions were 
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iy TABLE 1 
Correlation Matrices: Seven Behavior Rating Scales 


“The group” “The leader” 
1 2 3 4 5 6 1 2 3 4 5 
55 82 
-21 13 82 74 
—05 24 58 43 09 24 
-16 16 76 73 —25 -49 —50 38 
-14 -13 61 38 42 —24 -13 -37 15 58 
-18 —03 71 83 80 49 05 —06 09 50 53 
“One member” ay. 
1 2 3 4 5 6 1 2 3 4 5 
39 39 
-13 —30 17 36 
-11 22 42 20 40 62 
18 —02 -14 —02 60 61 41 33 
17 19 17 63 —44 36 13 54 37 36 
—63 —18 -14 18 09 —03 —04 04 48 75 36 


Note—Decimals omitted 


TABLE 2 
Unrotated Centroid Factor Loadings © 
“The group’ “The leader” 

Scale I SEV elit Lo Ee) ALL 

1. personal-impersonal 09 73 05 —63 70 —19 
2. internal-external 35 68 12 —73 51 26 
3. participation-nonparticipation 79 —33 —24 —72 53 —25 
4, interest-apathy 83 —13 29 22 65 —39 
5. sensitivity-insensitivity 83 —29 19 81 24 —09 
6. dominance-submission 53 —34 —39 60 29 38 
7. satisfaction-dissatisfaction 82 —44 28 50 59 14 
Za $ 309 151 43 276 19%4 49 

“One member” oye 

Scale I Il Wl Io a 

1. personal-impersonal ` 23 67 43 51 50 24 
2. internal-external 32 45 18 56 45 —36 
3. participation-nonparticipation 14 =30 —29 71 —27 -17 
4. interest-apathy 89 —37 —27 76 —34 —27 
5. sensitivity-insensitivity 04 —24 68 73 43 21 
6. dominance-submission 60 28 —62 62 —21 33 
7. satisfaction-dissatisfaction —03 —56 —15 63 —56 17 
RS EE E ST, o 
La 133 133 118 297 118 47 

Note—Decimals omitted 


"Extracted first 


—— ss E 
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TABLE 3 
Rotated Factor Loadings 

“The group” “The leader” 
ee an ea Se iranar 
Scale Bc ehh: pee I m im 
1. personal-impersonal —19 71 05 05 90 —19 
2 Premal external 07 76 12 aR = a 

3. participation-nonparticipation 85 —-Ol —24 -1 - 
4. interest-apathy 82 19 29 62 23 —39 
5. sensitivity-insensitivity 8 04 19 73 —39 —09 
6. dominance-submission 62 -12 —39 63 —16 38 
7, satisfaction-dissatisfaction 93 -10 28 77 03 —14 
xa’ 346 114 43 196 176 49 

“Qne member” Tah 

ph See ee SI a 
Scale ta a aA i Seah 2 3 
1. personal-impersonal E 79-27 06 71 24 
2. internal-external 18 53 —14 » 13 71 —36 
3. participation-nonparticipation 26 —30 17 70° 27 -17 
4. interest-apathy 80 02 57 79 24 —27 
5, sensitivity-insensitivity —40 30 45 28 80 21 
6. dominance-submission 88 04 —20 60 25 . 33 
7. satisfaction-dissatisfaction 00 —44 38 84 00 17 
Ea 367 1283 383 228 1% 4 


Note—Decimals omitted 


such that an interpretation was attempted. The analysis below can 
be followed by referring to the graphical or tabular solutions. 


Interpretation of Factors 


The most striking feature of the four factor plots was the appear- 
ance of two comparable factors from each get of scales. Thus, one 
factor was defined by Scales 1 and 2 on all four plots and can be 
referred to as the degree of personal reference factor. That is, a 
social object may be described at the personal, internal level, or it 
may be described in terms of those characteristics which are ex- 
ternal and can be objectified in an impersonal manner. 

The second factor was identified as degree of involvement and 
tended to be defined by the other five scales. For example, “The 
group” was seen either as being composed of interested, assertive, 
satisfied participants or of members who tended to withdraw in 
apathetic dissatisfaction. 
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Figure 1. Plots of First Two Rotated Factors for Various Stimulus Objects. 
| Another important feature of the four factor solutions was the 
$ manner in which some scales switched their factor association de- 
pending on which social object was being rated. Thus, for the rat- 
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ings of “The leader” statements, the scales covaried as described 
above with the important exception that the participation-non- 
participation scale (Scale 3) was now associated with the personal 
reference factor. The implication seems to be that when the group 
leader participated, it was on a personal level—he expressed per- 
sonal opinions, or gave subjective impressions of events, or referred 
to internal events in himself or the group. For the leader, then, 
participation in the group was an interpersonal process. Discussion 
of impersonal (intellectual) or external (out-of-field) topics by him 
was seen as nonparticipative in the interpersonal sense. 

For the “I”-statement ratings, the two-factor pattern again had 
one deviant element. This time, it was the sensitivity-insensitivity 
scale (Scale 5) which became associated with the personal reference 
factor. This says that more sensitivity was involved in referring to 
one’s internal self than to one’s external self. Although this is not 
a surprising relationship, it points up the fact that members did not 
associate sensitivity with personal expressions of the leader or other 
members. 

The factor pattern derived from “One member” statements is 
least similar to the other three patterns. One explanation would be 
that the variability of the social object (any member in each group 
meeting) led to a more complex set of associations than was evoked 
by the other descriptions. However, this explanation would not ac- 
count for the lack of complexity among the parameters of the “I” 
statements since these statements also included references to all 
group members, Apparently there is greater commonality in self- 
perceptions than there is in other-perceptions. At any rate, the 
personal reference factor for these statements carried a negative 
loading for Scale 7 which meant that members were seen as dis- 
satisfied when they expressed personal content rather than satisfied; 
or alternatively, statements were more often written about the 
personal references of other members when those references were 
negative than when they were positive. 

A negative loading also appeared on the involvement factor, this 
time on the senstivity-insensitivity scale. Thus the involvement of 
other members was perceived as varying from insensitive assertive- 
ness to self-conscious restraint and contrasted with the more sen- 
sitive interactions of the leader and the group as a whole. This pat- 
tern is suggestive of the distinction which has been made by Carter 
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(1954) and others between “self-oriented” and “group-oriented” 
behaviors. 


Factorial Time Trends 


In the previous section on scale trends over time, it was noted 
that the scales had been examined graphically and that because of 
the appearance of considerable covariance among the seven scales, 
it had been decided to factor analyze the scales before attempting 
to assess meeting-to-meeting trends. Thus the factor analyses re- 
vealed that a major portion of the scale variance over time could 
be accounted for in terms of two factors, one a content factor and 
the other a process factor. This finding permitted the analysis of 
time trends to proceed in terms of factorial constructs characterize 
by greater parsimony and reliability than was held by the sep- 
arate scales, 

To assess the time trends of the personal reference factor, new 
average ratings of each social object were calculated for each meet- 
ing based on the summed ratings of those scales which had high 
loadings on that factor. This meant that the average ratings of 
statements on “The group” and on “One member” were based on 
Scales 1 and 2; “The leader,” on Scales 1, 2, and 3; and “I,” on 
Scales 1, 2, and 5. Graphs of these averages are presented in 
Figure 2. 

The same procedure was applied to the involvement factor, and 
in this case the factorial rating of “The group” was derived from 
Scales 3, 4, 5, 6, and 7; “The leader,” from Scales 4, 5, 6, and 7; 
“One member,” from Scales 4 and 6; and “I,” from Scales 3, 4, 6, 
and 7. Graphs of these trends are presented in Figure 3. 
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Figure 2. Factorial Time Trends: Personal Reference Factor. 
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Figure 3. Factorial Time Trends: Involvement Factor. 


No adequate statistic is available to describe data of this nature. 
However, a subjective analysis was made that led to descriptions 
of group movement which can be compared with results of other 
studies. This analysis resulted in the following descriptive summary 
of this training group: 

(1) While involvement by the group as a whole was character- 
ized by an approach-withdrawal cycle throughout the life of the 
group, personal references increased gradually towards a peak at 
the seventh session and then oscillated for the remaining sessions. 

(2) The leader’s behavior was constant throughout except for a 
peak of personal referencing during the fourth session. 

(3) While other members’ involvement showed little meeting- 
to-meeting change, their personal references oscillated strongly dur- 
ing the second half of the life of the group. 

(4) Persons saw themselves as gradually becoming more and 
more involved during the life of the group. After the first session, 
their personal references were at a high level throughout, except 
during the fifth and sixth sessions. 


Discussion 
Methodology 


An important formal dimension of social-phenomena observer 
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systems has to do with the nature of the units which are used to 
identify and describe temporal aspects of behavior. Heyns and 
Lippitt (1954) have pointed to the difference between natural and 
imposed units and have noted that observational techniques were 
moving away from a concern with exhaustiveness and high inter- 
rater reliability and toward selectiveness, inferential observation, 
and theoretical relevance. More recently, Barker (1963) has dis- 
tinguished between subject-generated behavior units whose bound- 
aries are independent of the operations of the investigator and 
behavior tesserae which are imposed upon the behavior stream in 
accordance with an investigator’s scientific aims and preconcep- 
tions. 

In the present study, the basic behavior descriptions were made 
by members themselves without the imposition of a predetermined 
category system. Although statements were generated from four 
sentence stems as a means of focusing attention upon each relevant 
social object in the environment, members were free to make com- 
ments in terms of their own criteria for unit formation. These com- 
ments, then, are examples of natural units of content, although the 
attention of the observers was directed rather than free. 

A related characteristic of the present data system has to do with 
the criteria of selectivity which participants are presumed to have 
employed. In a broad sense the principle of selection may be said to 
be that of perceptual significance. As Hare (1962, p. 81) has pointed 
out, social perception is a complex phenomenon which is a function 
of cultural attitudes, the perceiver’s personality, the personality of 
the perceived object, unique situational factors, and the action at 
the moment. Efforts to isolate the effects of these various factors 
have not met with great success, and from a holistic point of view 
(e.g, Lecky, 1951; Combs and Snygg, 1959), such isolation is of 
dubious worth even when accomplished. The experimenter’s values 
and purposes are crucial at this point. He must ask if it is more 
important to test how well observers can use the experimenter’s 
category system or to permit observers to respond in terms of their 
own frame of reference. Should he guide observers to a predeter- 
mined conceptual viewpoint or permit them to discover their own 
perceptual commonality? Obviously, different answers to these 
questions should lead to different research designs, and examples of 
both should be found in the literature. However, the dependence of 
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social scientists upon the experimental methods of the physical 
scientists has meant an absorption of their values also, and most 
research in the behavioral sciences has tried to justify itself by the 
standard of objectivity. Fortunately, the recent development of 
new methodologies promises to free social science research from the 
tyranny of the correlation coefficient (Coombs, 1963; Shepard, 
1962), and nonmetric multidimensional analyses of interpersonal 
data have begun to appear (Rinn, 1963). 


Phenomenology and Education 


The decision to encourage idiosyncratic responses has both gen- 
eral theoretical grounds and specific educational purposes. Within 
a phenomenological framework, the explication of phenomena is es- 
sentially an inductive process whereby the reality of any phenom- 
enon is approached through a systematic attempt to observe and 
describe the world of experiential reality as it is presented to the 
person. This theoretical framework is congruent with the process- 
oriented approach to group learning which has most recently been 
summarized by Bradford et al. (1964). The major principles of 
this approach are a focus on the phenomenal field of the here- 
and-now and the development of member sensitivities to social 
phenomena; the discovery and utilization of yarious methods of in- 
quiry, including involvement, observation, feedback, and experi- 
mentation; the development of member skill in the processes of 
group membership, decision-making, self-assessment, and problem 
solving; and the transition from a group technology based on sus- 
picion, facade, manipulation, and dependency to one based on 
trust, openness, permissiveness, and interdependence. 


Cooperative Research 


It is important to note that the participant-observers in the pres- 
ent small-group, cooperative-learning, process-oriented setting were 
able to write descriptive statements beginning with four sentence 
stems and that this much structuring of descriptions was not re- 
sented by members who had opposed almost all other attempts to 
initiate structure into the group’s procedures. The advantages of the 
sentence-stem method are that it elicits responses to each element 
in the social field, the responses are in a form which is easily adapt- 
able to content analysis procedures, the method interferes mini- 
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mally with the ongoing process itself, and the method is congruent 
with educational goals which stress the active development of per- 
ceptual, descriptive, and interpersonal skills. 

Since the educational process included the value of cooperative 
development of research procedures, it would have been inconsistent 
to perform a content analysis of the data using externally derived 
scales. Thus, the criterion of participant significance was applied to 
the statement-rating process as well as to the observational process 
which produced the statements. Obviously, other rating scales could 
still be applied to the descriptive items, but if they are applied by 
nonparticipating raters, then any interpretation of ratings must 
be made in nonparticipative terms. Such interpretations can be 
useful for some purposes, but their relationship to the findings of 
the present study is that of a distant cousin, twice removed. 

A final aspect of the cooperative-research model is the feedback 
of data to the members of the project. In the present study this feed- 
back took two forms: immediate summaries of member statements, 
and a final summary of member ratings of statements over the 
series of eleven meetings. The final summary was mailed to mem- 
bers after the semester ended and consisted of the graphs of class 
averages of the seven scales for each sentence-stem category (28 
graphs). Each member also received his own ratings which he could 
analyze for time trends or compare with the class averages. 

It appears that this final step is an important one if members are 
truly to learn that the research process can be one of participants 
operating with each other rather than of being operated upon. Stu- 
dents are not only unlikely to encounter nonmanipulative designs 
in the research literature, but the idea is likely to remain an 
idealistic fiction unless they fully participate in the procedures 
implied by a cooperative philosophy. 


Group Dimensions 


A major part of the variance associated with ratings of descrip- 
tive statements of the various components of the social field can be 
understood in terms of two orthogonal dimensions: degree of in- 
volvement (both verbal and emotional) and degree of personal 
reference. It was as if members were asking two sets of questions 
about the group process. One set included such concerns as: Are 
we going to work closely together, or keep our distance? Will we be 
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sensitive enough to give and take, or will the powerful ones take 
over? Will the interaction be an interesting and satisfying one, or 
dull and unproductive? These concerns appear similar to those 
which have been subsumed by Schutz (1958) under the heading 
inclusion. They are also similar to those included in an involvement 
factor which was extracted by Rinn (1963) from free-response 
descriptions of the early meetings of a graduate course in counseling 
psychology. As in the present study, categories for a content analy- 
sis were supplied by the students themselves, Although categories 
were not developed cooperatively, the set of statements was divided 
by each student into several idiosyncratic categories of “similar- 
ity.” Frequency of joint inclusion in a category for each pair of 
statements provided a matrix of percentages which was analyzed by 
Coombs’ (1963) nonmetric multidimensional unfolding procedures. 

The personal reference dimension can be understood as repre- 
senting a group concern which may be characterized by another set 
of questions: Are we going to talk about our selves or about our 
ideas? Will we focus on the here-and-now or the there-and-then? 
Will we risk personal exposure or keep ourselves out of the spot- 
light? These concerns have to do with the content of interaction 
rather than the form (Hare, 1962, p.12). They suggest that a basic 
polarity which can split a group is that which divides members 
into those who accept the self as the central variable in social in- 
teraction from those who prefer to explain behavior in terms of 
environmental or other influences. Benne (1964) has provided an 
excellent discussion of the manner in which groups struggle to re- 
solve this and other paradoxes of group life. 


Time Trends 


Involvement. The time-trend analysis of the involvement factor in 
the present study revealed that members saw different involvement 
patterns for the various social objects. Thus, the temporal stability 
of the leader contrasted with a gradually increasing involvement of 
the self and also with a fluctuating cycle of approach and with- 
drawal by the group as a whole. The stable role of the leader was to 
some extent a result of deliberate self control, but the increasing 
self involvement of members would seem to reflect a steady growth 
in member confidence and spontaneity. 

The approach-withdrawal cycle of the group as a whole deserves 
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special consideration as evidence of a phenomenon which has not 
previously been observed at the group level. The phenomenon has 
been utilized at the individual level by the Gestalt therapists Perls, 
Hefferline, and Goodman (1951), however. Perls sees the rhythm of 
contact and withdrawal as being a necessary process in the creative 
adjustment of the self to the world, and he has devised a series of 
therapeutic techniques whose purpose it is to bring the patient into 
harmonious adjustment with this rhythm. 

If the existence of the ebb-and-flow phenomena should be veri- 
fied, it would seem to have significance for training-group theory 
and practice. From a group member's point of view, the phenom- 
enon would appear as a heteronomous event, to use Angyal’s (1941) 
term, which contrasts with autonomous happenings having their 
source in the person himself, Growth in member competencies is 
currently conceptualized within a framework which treats heter- 
onomous difficulties as temporary environmental conditions which, 
as Heider (1958) points out, are commonly diagnosed as due to op- 
portunity or luck. These conditions tend to be seen by group train- 
ers as disturbing the steady growth of member potentials and are 
dealt with as process problems or blocks to group productivity. An 
alternative conceptualization implied by the present findings is that 
the phenomena of intrapersonal growth and interpersonal inter- 
action have different natural patterns. This implies that interper- 
sonal contact and withdrawal should be dealt with as necessary 
phases of a natural polarity rather than in terms of a valued 
capacity for involvement and a devalued tendency for resistance 
and defensiveness. Involvement and detachment would then both 
be seen to have their unique values, and group trainers could deal 
with the phenomena descriptively rather than judgmentally. 

Personal reference. Turning now to the second factorial time 
trend, it can be noticed that the personal reference factor showed a 
different pattern for perceptions of the leader and self perceptions 
than it did for perceptions of the group and other members. Except 
for a high point during the fourth session, the leader’s contributions 
stayed at a constant moderate level of personalness, After the first 
session, members also rated their own contributions at a constant— 
although slightly higher—level except for the fifth and sixth ses- 
sions. The generally higher level of personal reference by “oneself” 
is easily accounted for in terms of the viewpoint of the observer— 
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whatever I say is more personal to me than what you say. The dip 
in the fifth and sixth sessions, however, may be related to the lead- 
er’s peak in the session a week previous. It may be hypothesized 
that intense personal referencing by the leader leads to a temporary 
retreat to impersonal content by members, although this hypothe- 
sis should be put to experimental test. If the relationship should be 
found to hold, its explanation could lie in the fact that the observer 
of an intense personal expression frequently feels more residual 
embarrassment and self consciousness than the immediate partic- 
ipant, possibly because the observer tends to be blocked off from 
active engagements which might serve to resolve his tensions. Goff- 
man (1959) suggests that the observer’s discomfort may be due to 
a disturbance of expectations regarding the actor’s role, and “that 
even sympathetic audiences can be momentarily disturbed, shocked, 
and weakened in their faith by the discovery of a picayune dis- 
crepancy in the impressions presented to them” (Goffman, 1959, 
p. 51). By this interpretation, members in the present study ex- 
pected the leader to maintain a low level of personal reference, were 
discomfitted when he did not, and turned for a time to an imper- 
sonal content level. 

The dips on the personal reference factor for ratings of other 
members can mostly be accounted for in terms of the above ex- 
planation. Thus, the dip which occurred during the fifth session 
followed the leader’s peak in the fourth meeting, while another dip 
in the eighth session followed a peak by the group in the seventh 
meeting. 

The degree of personal reference for the group as a whole followed 
the interesting pattern of gradually increasing for seven sessions 
and then fluctuating back and forth for the final four meetings. 
Having accepted the goals of involvement and personal reference, 
it appears that this group reached its maximal level of personal 
intensity during the seventh meeting and then began a “clinging- 


off” period in preparation for a return to less process-oriented work 
environments, 


Conclusion 


Tf one is interested in determining the frequency with which state- 
ments of various types are made, or in getting an “objective” record 
of member behaviors in a group, then the methodology which is 
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illustrated in the present study is inappropriate. However, if one 
takes a phenomenological approach to the study of social behavior, 
or if one is concerned with the felt importance of events to persons 
rather than with their frequency, then it is sensible to permit each 
subject to select those events that have significance for him. The 
record that is obtained under the latter condition is one of subjec- 
tive value rather than one of objective frequency. When such data 
are summed and averaged, as in the present study, any relation- 
ships that can be shown to exist have the promise of truly repre- 
senting group phenomena. Although the hypotheses that are de- 
rived from such relationships are likely to be weak, they are also 
likely to have great generality. . 
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SECULAR TRENDS IN AN ADJECTIVE CHECKLIST 
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Royal Netherlands Navy? 


Wuen forced-choice personality inventories are constructed on 
the basis of item indices (such as favorability or popularity) as ob- 
tained through administration of the items in some rating-scale 
format, one assumption implicitly made by most test constructors 
is that these indices are not biased appreciably by the position in 
which the items occurred. More generally, such an assumption is 
implicit in any selection process whereby items are chosen on the 
basis of certain characteristics, and where position has not been 
controlled in assessing these characteristics. 

Gordon (1952), among others, demonstrated that position does 
influence response to items in long personality inventories. By 
systematically varying item position, he was able to show that sub- 
jects tend to respond in a more desirable manner as the list pro- 
ceeds, One of Gordon’s conclusions was that some method of ro- 
tating the items would be desirable when long lists of items are 
presented for the purpose of obtaining preference indices for use in 
the construction of forced-choice inventories. 

In a recent study (Stricker, 1964) that reviews most of the lit- 
erature on secular trends (cf. Loevinger, 1957, p. 676 ff.), a similar 
finding with respect to criticalness is reported: the extent to which 
an item measured such a response style was found to be related to. 
its location in the test, while the content validity of the item was 
not predictable from its location. 

1The author wishes to thank Drs. Lawrence Stricker and Richard Melton 
for their comments on this paper. 

2 The present investigation was completed while the author was a fellow of 


the Netherlands Organization for the Advancement of Pure Research AS W.O.) 
at Educational Testing Service. 
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The term “secular trends,’ which was originally (Loevinger, 
1957) proposed to account for systematic changes that take place 
upon readministration of a test or its parallel form, and which was 
subsequently broadened to subsume such changes within a single 
administration (Stricker, 1964), could be further expanded to in- 
clude even shorter-term trends such as order and context effects, 
described by Cowen and Stiller (1959). The authors administered 
a list of 30 neutral trait-descriptive adjectives in counterbalanced 
order; subjects were to rate the traits for desirability. Context ef- 
fects appeared in that positive adjectives rated first were seen as 
less socially desirable than in the setting in which item indices had 
been obtained; neutral adjectives were seen as relatively more de- 
sirable when rated first. Order effects appeared to the extent that 
positive adjectives became more desirable when rated after neu- 


trals, while the reverse order decreased the desirability of neutral 
items. 


Psychometrically, the situations just described are similar in 
that item response is to some extent dependent upon whether or not 
subjects have answered other items of the same sort; whether the 
basis for such sequential redundancy be fatigue or boredom, changes 
in the subjects’ cognitive or emotional approach to the test, or 
changes in adaption level (Cowen and Stiller, 1959). It might be 
profitable theoretically to reserve the term “secular trends” for the 
effects of any such changes that are produced by experience with 
the test itself, as opposed to learning outside the testing situation. 

The present research was undertaken to assess the influence of 
secular trends upon response to a list of trait-descriptive adjectives, 
insofar as such trends result in biased estimates of popularity in- 
dices. Two hypotheses were tested: (a) popularity indices of items 
which occurred later in the list would turn out to be overestimated 
in the case of positive adjectives, and underestimated in the case of 
negative traits; (b) the popularity value of a trait is influenced by 

* the popularity of the trait immediately preceding it. 


Method 


A list of 180 trait-descriptive adjectives was administered to 120 
Dutch military subjects. Instructions were to indicate on a five- 
point scale the extent to which a trait was applicable. Responses 
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were weighted 1 to 5 and averaged for each trait to provide popu- 
larity indices (PI). 

On the basis of these PI, forced-choice formats were constructed. 
These formats consisted of tetrads, each having a pair of popular 
and a pair of less popular traits. The two items forming a pair were 
closely matched on PI. Instructions for the forced-choice format 
were to indicate, for each tetrad, the trait that was “most appli- 
cable” and the trait that was “least applicable.” 

Two forms were constructed. Form A consisted of 20 tetrads; it 
was administered to a sample of enlisted men in the Netherlands 
Navy, of which 100 were chosen randomly for the analysis. Form 
B consisted of 36 tetrads and was given to a sample of Dutch army 
men, of which 476 were selected randomly. 

For each item (80 in Form A, 144 in Form B), “most applicable” 
and “least applicable” responses were tallied, and the latter num- 
ber was subtracted from the first. The item index thus obtained will 
be referred to as DI (difference index); it is, however, merely 
another popularity index, obtained in a different context. 

For each pair of items which were matched on PI, a comparison 
was made between the two DI, and it was attempted to predict the 
direction of the difference between the two DI on the basis of 
hypotheses (a) and (b). Predictions were (a) that for a pair of 
popular items (items with PI > 3.00), the member that occurred 
later in the original list would have the lower DI (because its PI 
would be inflated); and that for a pair of unpopular items 
(PI < 3.00), the member that occurred later in the rating list 
would have the higher DI; and (b) that for each pair of items, that 
were matched on PI, the direction of the difference between the two 
DI would be systematically related to the direction of the differ- 
ence between the PI of the single items which immediately pre- 
ceded the members in the original list. 


Results and Discussion 


Hypothesis (a), concerning a growing set to respond desirably in 
a long list of items, was not substantiated: of the 40 predictions 
that were made for Form A, 21 were in the right direction and 19 
were wrong; for the 72 pairs of Form B, 38 predictions were right 
and 34 were wrong. The position effect as demonstrated by Gordon 
(1952) does not seem to have been particularly influential.: There 
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may be at least two reasons for the lack of practical significance of 
this secular trend in the present case. In the first place the present 
list, even though containing numerically as many items as 
Gordon’s, was in all likelihood shorter as far as testing time is con- 
cerned: adjectives require less reading time than statements. Sec- 
ond, the influence of other effects may have been so powerful as to 
obscure position trend. 

Hypothesis (b), representing the influence of the previous item’s 
desirability on PI, was clearly confirmed by the results, to the ex- 
tent that the PI of the members that were preceded by more de- 
sirable items turned out to be overestimated ; consequently, these 
members had the lower DI within a matched pair. With this direc- 
tion in mind, 25 out of 39 predictions were right for Form A 
( = 3.10, p < .10), and 49 out of 71 were right for Form B 
(x? = 10.27, p < .01). In both cases, one prediction could not be 
made because of a tie in PI. 

In view of the indirect method employed, it is somewhat sur- 
prising that the effect of the preceding item upon trait popularity 
could be so clearly demonstrated. The effect seems to be of con- 
tamination: the PI of an item is biased in the direction of the pre- 
ceding item. Subjects appear to perseverate in responding to a list 
of traits, whether such perseveration is of motor or of more central 
nature, 

In a sense, the present findings are opposite to the order effects 
found by Cowen and Stiller (1959). While in their study a con- 
trast effect appeared, the present results point toward assimilation, 
Presumably, the difference is one of design: while Cowen and Stiller 
used lists that were very homogeneous with respect to social de- 
sirability, the present study employed a very heterogeneous list. 
It seems plausible that a homogeneous list should produce con- 
trasting tendencies, while a sequentially heterogeneous list should 
elicit, perseveration. 

The results of this investigation illustrate once more the neces- 
sity of taking secular trends into account; if the broader definition 
here proposed is accepted, it would be hard to maintain that “the 
‘theory of the first test? can ignore them” (Loevinger, 1957; 
p. 678). Furthermore, the study of secular trend may well contrib- 
ute to further insight into what subjects are actually doing when 
they fill out a personality inventory. At least in personality testing, 
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the notion of an isolated response to an isolated item seems to be- 
come less and less realistic. 


Summary 


Secular trends in tests were redefined as effects of systematic 
changes in response to test items resulting from experience with 
similar items, rather than from learning outside the testing situa- 
tion. In the present study, two such trends were investigated in a 
list of 180 trait-descriptive adjectives: a trend for items to be re- 
sponded to more desirably as the list continues, and a trend for 
items to be responded to more or less desirability depending upon 
the desirability of the preceding term. The method consisted of 
comparing item popularities, as found in the original format, with 
the popularities of the same items presented in forced-choice for- 
mat: if item popularities had been biased through the operation of 
secular trends in the original list, such bias should to some extent 
be detectable in the responses to the forced-choice format. The re- 
sults showed no evidence for the first trend; with respect to the 
second, it was clearly demonstrated that items in the original list 
were responded to more desirably as the preceding item was more 
desirable. The importance of controlling and studying secular 
trends was discussed. 


REFERENCES 


Cowen, E. L. and Stiller, A. The Social Desirability of Trait De- 
scriptive Items: Order and Context Effects. Canadian Journal 
of Psychology, 1959, 13, 193-199. 

Gordon, L. V. The Effect of Position on the Preference Value of 
Personality Items. EDUCATIONAL AND PSYCHOLOGICAL MEASURE- 
MENT, 1952, 12, 669-676. 

Loevinger, Jane. Objective Tests as Instruments of Psychological 
Theory. Psychological Reports, 1957, 3, 635-694 (Monograph 
Supplement, 1957, No. 9-V3). 

Stricker, L. J. Difficulty and Other Correlates of Criticalness Re- 
sponse Style at the Item Level. EDUCATIONAL AND PSYCHOLOGICAL 
MEASUREMENT, 1965, 25, 683-706. 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
_ Vou, XXVI, No. 2, 1966 
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Harvey, Hunt, and Schroder’s (1961) classification of individ- 
uals according to the level of integrative complexity of their person- 
ality structure has led to the accurate prediction of the individual’s 
diversity of perceptions (Driver, 1962), reaction to attitude refu- 
tation (Streufert, 1962), and attitude change in a sensory deprived 
and isolated environment (Suedfeld, 1964). The classification sys- 
tem, when used to compare homogeneous groups, has made possible 
the prediction of sensitivity to feedback (Lawrence, 1962) and 
group structure and performance on a variety of measures in a 
simulated environment (Tuckman, 1964). This evidence provides 
validity for the classification scheme and its underlying theory, as 
well as the measure used for identifying the individual's level of 
integrative complexity. The measure used for this purpose has been 
the Sentence Completion Test (SC) developed by Schroder and 
Streufert (1962), a projective instrument, which can be considered 
to have been validated by the above studies. 

The purpose of this study was to evaluate an objectively-scored 
device to measure integrative complexity by determining the ex- 
tent to which it was useful in predicting creativity, a characteristic 


1 From Bureau of Medicine and Surgery, Navy Department, Research Task 
MR005.12-2005.01, Subtask 1. The opinions and statements contained herein 
are the private ones of the writer and are not to be construed as official or re- 
flecting the view of the Navy Department or the Naval Service at large. 

The author would like to thank Mr. Robert Nye for his help in collecting 
the data and Dr. Irwin Altman for his valuable suggestions in analyzing and 
interpreting the data. 
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implied in the concept of integrative complexity, and to determine 
further whether this new instrument classified individuals similarly 
to the SC. Since the SC, as a projective instrument, is difficult to 
score reliably, and scoring must be preceded by the training of 
scorers which is arduous and time-consuming, an objectively- 
scored measure of integrative complexity would be useful. A fur- 
ther advantage would be gained in dealing with persons of limited 
verbal ability who respond tersely and incompletely to open-ended 
questions, 

Four nodal systems of integrative complexity have been de- 
scribed in detail by Harvey, et al. (1961). 

System I. At the lowest level of integrative complexity, the rules 
or schemata for categorizing stimuli are highly fixed and simple. 
Ambiguity is not tolerated and simple schemata, norms, or authori- 
ties help the individual to structure his environment in a complete 
and unyielding way. System I individuals are characterized by 
categorical, black-white thinking, minimization of conflict and 
avoidance of ambiguity, self definition in terms of external an- 
chors, preservation of standards and minimization of alternatives, 
and overgeneralization of fixed approaches or stereotypes. 

System II. At a level of integrative complexity somewhat above 
that of System I, the schemata for categorizing stimuli are still 
relatively simple, but more alternatives are perceived than in the 
case of System I functioning. The System II individual perceives 
his world against a background of self vs. other (representing the 
separation of self and other which is beyond the integrative com- 
plexity of System I), and accepts self while rejecting other. This 
leads to an absolutistic orientation toward others who, when seen in 
a position of potential control are “warded off;” one experiences 
conflict when external standards are imposed on the self. This de- 
tached, antidependency characterization has been termed a nega- 
tively independent orientation. 

System III. At this higher level of integrative complexity the 
schemata for “reading” the environment are more flexible and more 
alternatives are perceived. Not only is the self highly differenti- 
ated but other people are equally differentiated. This latter fact 
enables the System III to be highly sensitive to others and to at- 
tempt to match his perceptions to those of others. That is, he is 
highly capable of putting himself in the role of others and perceiv- 
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ing himself as others perceive him. The orientation, then, is toward 
the maintenance of close interpersonal relationships and rejection 
is threatening. 

System IV. At the highest level of integrative complexity a di- 
verse world filled with many alternatives is perceived. The System 
IV individual uses highly complex and flexible schemata for read- 
ing his environment and those in it. Interpersonally, this individual 
is highly autonomous and reacts to people as a source of informa- 
tion. The System IV person generates a large variety of alternative 
interpretations of environmental events and can thus react to the 
subtleties of his environment with appropriate and novel responses. 


Problem 


Generalizing from the descriptions of the four systems, it was 
decided that the following criteria would be used in evaluating an 
objectively scorable measure of integrative complexity, henceforth 
designated as the Interpersonal Topical Inventory (ITI) :? 

1). The system classification of an individual based on his profile 
of scores on the ITI should make possible the prediction of his per- 
formance on a battery of tests of creativity following the assump- 
tion that the more integratively complex the individual, the more 
creative he will be. 

The anticipated relationship between integrative complexity and 
creativity was based on the fact that the former is a measure of the 
extent to which the individual separates (i.e., differentiates) and 
recombines (i.e., integrates) inputs in a variety of ways, while the 
latter is a measure of the extent to which the individual’s thinking 
embraces novelty and speculation and leads to the production of 
unusual but relevant responses. Integrative complexity describes 
the structure of the individual’s life space in terms of its degree of 
differentiation and integration while creativity describes the char- 
acteristic responses produced as a function of the individual’s level 
of integrative complexity. The more alternatives the individual ap- 
prehends in his thinking, the more likely he is to produce novel, 
creative responses. Moreover, the less anchored the individual’s 


2 This is not a test of validity in the traditional psychometric sense. Rather, 
it ig an attempt to evaluate the research usefulness of a new, objectively-scored 
instrument for classifying people. Consequently, test-retest reliability and in- 
ternal analyses characteristic of a thorough-going psychometric effort will not 
be presented. 
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thoughts are in the external situation and external criteria, the 
more likely they are to be creative. Since integratively complex 
persons produce more alternatives and are less externally con- 
strained, they were expected to be more creative, i.e., produce more 
creative responses. 

If the ITI is a useful measure of integrative complexity, individ- 
uals classified by the ITI into systems should be rank ordered 
System IV, System III, System II, System I on creativity with IV’s 
most creative. The system classification of the individual based on 
his scores on the SC should lead to the same rank-ordering of the 
system groups on creative performance as that for the ITI. 

2. The system classification of individuals based on ITI profile 
scores should correspond to classification on the SC scores. If classi- 
fication using the ITI corresponds to classification using the SC, 
the more objective ITI measure will then be considered a poten- 
tially useful substitute for the SC as a measure of integrative com- 
plexity. 


Method 
Subjects 


Subjects were 126 Navy enlisted men from the U. S. Naval Serv- 
ice School Command, Bainbridge and the U, S. Naval Station, 
Washington, D. C. The median age of Ss was 18 with a range of 
17 to 24. Seventy-nine per cent of the Ss were high school graduates 
while 8 per cent had, in addition, some college experience. None had 
a college degree, 

Intelligence of the subject Population was assessed by the Navy 
General Classification Test (GCT) and only individuals having 
GCT scores of 53 (IQ = about 106) and over were included in this 


study. The 126 Ss had a median GCT score of 59 with a range of 
53 to 71. 


The Predictor Measure: The Interpersonal Topical Inventory 


The Interpersonal Topical Inventory of integrative complexity 
(ITI) is a forced-choice instrument in which S$ is asked to choose 


3 The author would like to thank CDR. A. D. Garvin, USN, din, 
officer of the U. S. Naval Service School Command, Bainbades, Md,-and 
Chief Ships Clerk W-3 J. F. Cullinan, separations officer at the U. S. Naval 
Station, Washington, D. C. for Providing the subjects used in this study. 
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one of a pair of items that best represents his feeling about or re- 
action to an interpersonal topic. The topics are: (a) when criticized, 
(b) when in doubt, (c) when a friend acts differently toward you, 
(d) beliefs about people in general, (e) feelings about leaders, (£) 
feelings about rules. These topics are meant to confront the indi- 
viduals with interpersonal conflict, ambiguity, and the imposition 
of control. For each topic there are six pairs of alternatives; S must 
pick one from each pair. Each member of a pair represents a typical 
response for a different system. All possible paired combinations of 
the four systems go to make up the six pairs. 

8 is given four scores, each representing his total number of 
choices for each of the four systems. The maximum score for any 
system is 18. Based on the frequency distribution of system scores 
for the 126 Ss used in this study plus another 85 Ss run previously 
for other purposes, half-decile ranges (each containing five per- 
centiles) were identified and each of the four scores for each S was 
replaced by a percentile score and a half-decile score. Ss scoring 
in any one of the 8 highest half-deciles (i.e., scoring at the 61st. 
percentile or better) on one of the four systems and lower on the 
other three were assigned to that system.* Thus, system assignment 
was on a relative rather than an absolute basis. On this basis, 31 
Ss were classified as System I, 26 as System II, 22 as System III, 
and 30 as System IV. Only 17 Ss out of the 126 (13.5%) could not 
be classified because they scored equally high in more than one 
system or not high enough in any. 


The Sentence Completion Test of Integrative Complexity 


The Sentence Completion Test (SC), developed by Schroder and 
Streufert (1962), is a projective instrument in which S is asked to 


4TIf S scored within the same half-decile range on two systems and these 
scores were within the four highest half-deciles (percentiles 81-100), then ties 
were resolved as follows: 

1. If one of the systems was System I, S was classified into System I. 

2. If one of the systems was System IV and the other not System I, S was 
classified into System IV. 
Such ties occurred in 12 cases. In no case, were System II and III tied. The 
above rules were adopted since the systems at the extremes (i.e. I and IV) 
appear to be most clearly delineated by the ITI; and since they are most 
likely to be used as experimental groups, resolution of ties in their favor is an 
aid to classification. One outcome of the use of these rules was to reduce the 
number of Ss classified into Systems II and III relative to the other two sys- 
tems. However, since these rules made it possible to classify more people, they 
were deemed practical. 
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respond to stems which imply interpersonal conflict, ambiguity, or 
the imposition of control. The SC used in this study contained six 
stems. Each completion is scored on three scales: degree of abstract- 
ness, degree of System II content, and degree of System III content. 
The first scale is a five-point scale while the latter two are four- 
point scales. Inter-scorer reliabilities from .70 to .98 have been re- 
ported by Schroder and Streufert (1962). For this sample, the fol- 
lowing reliabilities (using Pearson’s Product Moment Correlation) 
were obtained: .80 for the degree of abstactness scale, .80 for the 
System II content scale, and .51 for the System III content scale. 

Following essentially the procedure outlined by Schroder and 
Streufert (1962), Ss were assigned to one of the four systems. On 
this basis, 35 Ss were classified as System I, 25 as System II, 19 as 
System III, and 27 as System IV. Twenty Ss could not be classi- 
fied (16%). In terms of reliability of classification (based on the 
profile of the three scale scores), 76 per cent of the rescored cases 
were classified as they had been originally. Moreover, about one- 
half of the disagreements involved cases originally designated as 
unclassifiable. 


The Creativity Test Battery 


The creativity battery consisted of the following tests: 

1. Gestalt Transformations (Guilford, et al., 1952). The subject 
is asked to choose an object from among five choices that has a part 
which will serve a specified purpose. None of the objects listed is 
ordinarily used or associated with the specified purpose which is to 
be served. The score was the total number of correct solutions. This 
test loads on the factor of “semantic redefinition” identified by 
Guilford and Merrifield (1960). 

2. Match Problems II (Berger, Guilford, and Christensen, 1957). 
The subject is asked to indicate several different patterns of matches 
that can be removed to leave a certain number of squares or tri- 


5 The author is again grateful to Mr. Robert N: i 
r; . ye for scoring the Sentence 
k Completion Tests. He was totally unaware of the names and identifications of 


tests to obtain reliability estimates. 
6 System assignment based on SC scores differed from th: 
at sted b 
Schroder and Streufert (1962) only insofar as ranges were defined ae id 
rather than an absolute basis, Rather than saying, a priori, that the System IV 
range on the degree of abstractness scale is 4.0-5.0, System IV range on this 
scale was defined here by Ss scoring in the top 20 Per cent on this scale, 
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angles. The score was total number of correct solutions. This test 
loads on the factor of “figural adaptive flexibility” (Berger, et al., 
1957). 

8. Consequences (Christensen, Merrifield, and Guilford, 1958). 
The subject is asked to list as many consequences as he can of cer- 
tain hypothetical and unusual situations. Two scores were given: 
one for total number of relevant responses, and one for number of 
remote responses. Remote responses were defined as those more dis- 
tant, temporally or geographically, than an immediate or obvious 
response. Inter-scorer agreement on judgments of obvious and re- 
mote on a randomly drawn sample of 220 responses amounted to 
95 per cent. 

This test loads on the factor of “originality” identified by Guil- 
ford and Merrifield (1960). 

The four scores obtained from the three creativity measures were 
converted to Z scores. The criterion was a composite score labelled 
the combined creativity score, which was obtained by summing the 
four Z scores: Gestalt Transformations + Match Problems + Con- 
sequences: Total -+ Consequences: Remote. The combined cre- 
ativity score is highly similar to the creativity criterion score used 
by Garwood (1964), the difference being that Garwood included 
three additional measures of creativity. A discussion of the validity 
of the creativity measures appears in Garwood (1961). 

A measure of creative motivation, the Creativity Motive Ques- 
tionnaire (Golann, 1962) was also included in the battery. This 
test presents the subject with a pair of tasks or situations, differing 
in the extent to which each will appeal to the creative motive, and 
he must indicate his preference. The score is the total number of 
“creative” tasks or situations chosen. 


Results 


The Prediction of Creative Performance 


The means on the combined creativity score were compared by 
analysis of variance and Duncan’s Multiple Range Test (Winer, 
1962) for each of the four system groups, as classified separately by 
the ITI and SC. These means, and the outcomes of the analyses 
appear in Table 1. 

As shown, means for the groups classified by the ITI were signi- 
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TABLE 1 
Means and Analysis of Variance of Combined Creativity Scores 


Systems (means) Analysis of Variance 

Test I I m EV. MSs df MSw df F 
ITI 

(Objec- 

tive) 38.55" 43.08 41.59 44.40 170.9 3 39.9 105 4.28* 
SC 

(Projec- 

tive) 38.09" 42.12° 42.78 43.93 168.9 3 34.9 102 4.63* 
*p < 01 


a Significantly different from other 3 means by Duncan Range Test (p < .01) 
> Significantly different from mean for System IV by Duncan Range Test (p < .01) 

° Same as b, but p < .05 
ficantly different (F = 4.28, p < .017) as were means for the groups * 
classified by the SC (F = 4.63, p < .01). In both cases, high F ra- 
tios were based primarily on differences between the creative per- 
formance of System I individuals versus all other systems. That is, 
using either instrument as a basis for classification, System I in- 
dividuals performed less creatively than did individuals of the 
other three systems. 

Among the other systems, only System IV and System III were 
significantly different from one another when the ITI was the basis 
for classification, while System IV and System II were significantly 
different from one another when the SC was the basis for classifi- 
cation. In the former case, II’s exceeded III’s and IV’s exceeded II's, 
but these differences were not significant. Using the SC, III’s ex- 
ceeded IT’s and IV’s exceeded III’s, but these differences were not 
significant. Thus, creative performance for system groups classified 


by both instruments was highly similar, and in the predicted direc- 
tion. 


The Prediction of Creative Motivation 


Mean Creativity Motivation Questionnaire (CMQ) scores and 
analyses for the system groups classified separately by the ITI and 
SC appear in Table 2. CMQ means of groups classified by the ITI 
did not differ (F = 0.75) while creative motivation of groups 
classified by the SC did differ (F = 415, p < 01). In the latter 
case, the high F ratio was based on differences between System I 


7 All probabilities listed in the text and the tables are two-tailed. 
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TABLE 2 
Means and Analysis of Variance of Creativity Motive 
Questionnaire (CMQ) Scores 
Systems (means) Analysis of Variance 
Test I IL Il IV MSs df MSw df F 
ITI 
(Objec- 
tive) 19.42 21.15 2041 21.63 25.1 3 33.7 105 0.75* 
SC 
(Projec- 


tive) 18.178 20.92 20.95 23.63" 1251 3 30.2 102 4.15** 


* Not significant 
p< .01 
a Significantly different from other 3 means by Duncan Range Test (p < .01) 


and the other three systems, and differences between System IV 
and the other three systems. Of the six comparisons made, only the 
difference between System II and System III means was not signi- 
ficant. Clearly, system classification via the SC led to the identi- 
fication of inter-system differences in creative motivation, while 
system classification via the ITI did not. These results were con- 
trary to what was expected in the use of the ITI. 


The Prediction of Intelligence 


The four system groups, as classified separately by the ITI and 
SC, were also examined with regard to intelligence (Table 3). As 
shown, the system differences in creative performance and creative 


TABLE 3 
Means and Analysis of Variance of Intelligence (GCT) Scores 
Systems (means) Analysis of Variance 
Test Li; yt Il IV MSs df MSw df F 

ITI 
(Objec- 

tive) 59.50 60.50 59.71 60.62 7.9 3 18.3 98** 0.43* 

Sc 
(Projec- 


tive) 58.45 60.72 59.94 61.28 35.2 3 168 95% 2.09* 


* Not significant 
** There are fewer degrees of freedom than in previous analyses because intelli 
not available for 9 Ss. ” telligence scores were 
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motivation could not be accounted for on the basis of intelligence, 
using either the ITI or the SC. 


Direct Comparison between the Two Classification Instruments 


Level of Abstractness as a Criterion. The Sentence Completion 
Test contains three scoring scales, one of which is a five-point level 
of abstractness scale. In using this scale to classify system-types, 
IV’s are considered to be more abstract than III’s, III’s more ab- 
stract than II’s, and II’s more abstract than I’s. The level of ab 


stractness score was used as a criterion for evaluating system classi- 
fication based on the ITI. 


TABLE 4 


Means and Analysis of Variance of Level of Abstractness Scores 
(taken from the Sentence Completion Test) 


Systems (means) 
Test I Ir m IV 
ITI 
(Objec- 


tive) 1.92 1.99 2.17> 2.468 


*p < 05 
^ Significantly different from other 3 means by Duncan Range Test (p < .01) 
»'Significantly different from mean for System I by Duncan Range Meet (p < .05) 


1.62 3 0.47 105 3.45% 


From Table 4 it can be seen that levels of abstractness for the 
groups classified by the ITI were different (F = 2.45, p < .05). This - 
high F-ratio was based on significant differences between System 
IV and each of the other three systems, and between System Tt 
and System I. Although the System II mean was higher than that ` 
of System I, and the former was lower than that of System III, as 
expected, neither difference reached acceptable levels of significance, — 

ITI VS. SO: person-by-person classification. The most direct 4 
test of agreement, between the two classification measures was to 
examine the extent of correspondence between classification made 


in Table 5. This frequency data yielded a X? = 38,02 (p < .01), 3 


coefficient (Guilford, 
indicating a strong 
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TABLE 5 


Cross-classification of Individuals by the Interpersonal Topical 


Inventory and the Sentence Completion Test 


Classification Based on the ITI 
Ii 


IV 

I i) [eee ae EN 32 

82 A i ue eg 21 
3 do 

S2 m E ae a e g, 18 

2 v 4 SAE SIS 23 

iS) 
2. «498 +#«20 25 94 


relationship between the two classification instruments. An ex- 
amination of Table 5 shows that inter-test agreement is at or be- 
yond 50 per cent for all systems except System III where the extent 
of disagreement is high. Here Ss classified as System III by the ITI 
are as likely to be classified as System I by the SC as they are to be 
classified as System III. 

Discussion 

Using the criterion of creative performance, System I Ss were 
found to do significantly worse than Ss from any of the other three 
systems regardless of whether the ITI or the SC was used as the 
system classification instrument. In general, I’s were found to be 
least creative, and IV’s most creative with II’s and III’s in-between. 
The results leave the location of Systems II and III with respect 
to each other and to the other two systems somewhat ambiguous. 
Based on the ITI as the classification instrument, III’s are substan- 
tially less creative than IV’s. Based on the SC as the classification 
instrument, II’s are substantially less creative than IV’s. The latter 
finding is more in line with expectations. 

The ambiguity can be somewhat resolved by examining the area 
of disagreement between the two classification instruments. Table 
5 shows that many individuals classified into System III by the ITI 
are classified into System I by the SC. If the ITI is weak in this 
area of classification, then these individuals are, in fact, misclassi- 
fied I’s. Since I’s are the least creative of all system types, if they 
are being erroneously included in the ITI System III group then 
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the mean for this group is being artifactually lowered. This would 
account for the transposition of II’s and III’s on creative per- 
formance when the ITI is the classification instrument. 

This conjecture was examined more systematically by comparing 
the creative performance scores for the seven Ss who were classified 
as System III by both instruments with the eight Ss who were 
classified as System III by the ITI and System I by the SC. For the 
former seven, five of whom had creativity scores of 40 or above, 
a mean creativity score of 42.7 was obtained. For the latter eight, 
three of whom had creativity scores of 40 or above, a mean cre- 
ativity score of 40.0 was obtained. While the number of Ss involved 
are not sufficient for a meaningful statistical test, it appears plausi- 
ble that some System I individuals were inaccurately classified as 
System III by the ITI, producing a spuriously low mean creativity 
score for System III. 

It was concluded that there is a strong relationship between cre- 
ative performance and level of integrative complexity based on the 
analysis of mean combined creativity scores for system groups iden- 
tified by the SC. It was also concluded that the ITI is a useful re- 
search instrument and reasonable substitute for the SC for identi- 
fying Systems I, II, and IV but is inadequate in its present form 
for identifying System III. This Conclusion was based on the 
predictability of creative performance when the ITI was used as the 
classification instrument and the extent of agreement between the 
two system-measures, However, the adequacy of the SC for measur- 
ing System III is also questionable owing to the low inter-scorer 
reliability obtained for the System III scale of the SC. 

Inter-system differences in creative performance were not expli- 
cable on the basis of intelligence since systems did not differ sig- 
nificantly in intelligence using either classification instrument. 
Intelligence effects were somewhat higher when the SC was the 
classification instrument but this is probably due to the fact that 
verbal ability plays a bigger role in writing an original response 
than choosing from among given responses. 

The SC as a classification instrument led to predictable differ- 
ences between systems on creative motivation while the ITI did not. 
It may be that the SC, as a projective, open-ended instrument, is 
more sensitive to individual motivation than is the ITI. However, 
the “price” paid for this heightened sensitivity is, in many cases, 
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excessive. The SC is difficult to score, and is especially limiting 
when rapid scoring is necessary. 

The ITI was designed to be an adequate, easily scored measure 
of the systems of integrative complexity, primarily for use as a re- 
search tool. Based on the findings of this study, it was concluded 
that the ITI significantly discriminated between systems, but that 
further efforts were needed to improve its capacity to measure 
System III. Such efforts should be based on criteria other than cor- 
respondence to the SC. 


Summary 


This study attempted to evaluate an objective measure of level of 
integrative complexity, the Interpersonal Topical Inventory (ITI). 
Evaluation was based on the extent to which the objective ITI 
could predict creative performance and creative motivation as well 
as scores on a projective instrument presently in use as a measure of 
integrative complexity, the Sentence Completion Test (SC). A fur- 
ther source of evaluative data was the correspondence between the 
new classification instrument and the projective SC. 

The concept of integrative complexity represents the extent to 
which the individual perceives his world and those in it in a highly 
differentiated and integrated manner. The greater the individual’s 
capacity for differentiation and integration, the more likely he is to 
produce creative responses. On this basis, creative performance was 
deemed an evaluative criterion for measures of integrative com- 
plexity. 

Data from 126 Naval enlistees showed that classification by 
either the ITI or the SC led to similarly accurate predictions of 
creative performance, as measured by a battery of creativity tests. 
This was not the case for creative motivation, where accurate pre- 
dictions followed only from SC classification. However, the degree 
of correspondence in classification of individuals by the two instru- 
ments was high at all levels of integrative complexity except one. 
In no case were differences in integrative complexity accountable 
for on the basis of intelligence. 

It was concluded that the objective ITI is a potential substitute 
for the projective SC in large scale surveying of individuals for 
research, 
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Tuns is a time in educational history when knowledge and know- 
ing are again being heavily stressed as goals for education, and 
when new curricula are being developed to achieve these goals, Si- 
multaneously, research on the “anatomy of teaching” and the “logic 
of teaching” is being conducted to examine what the teacher does 
when he teaches, without regard to the what and the why. It is im- 
portant, however, to examine a teacher’s‘philosophy of education, 
defined here as a logically inter-related set of concepts of reality, 
knowledge and values. This philosophy must be understood in con- 
junction with the analytical study of teaching in order to gain 
insight into just what the teacher views to be the goals of education. 
This study, therefore, was undertaken to develop a short, reliable 
and valid instrument to measure the ontological, epistemological 
and axiological dimensions of a teacher's philosophy of education. 

An inspection of research publications and the professional lit- 
erature since 1952 reveals the absence of any widely used instru- 
ments for measuring a teacher’s philosophy of education. Among the 
few efforts to develop a measurement instrument of this type is a 
two-philosophies (empirical-rationalistic) Q-sort instrument called 
the GNC (Gowin, Newsome and Chandler, 1961). E. V. Sayers de- 
veloped a paper-pencil test of consistent agreement with the edu- 
cational philosophy of experimentality and proceeded as far as 
item analyses and reliability tests on education student populations 
in Hawaii and New York City. 

Ryans (1961), as part of the Teacher Characteristics Study, de- 
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veloped an estimate of conservative educational viewpoint. This 
score measured the dimension called conservative, learning-cen- 
tered—liberal, child-centered. Ryans reports that scores on this 
twenty-item forced-choice instrument revealed differences between 
teachers related to grade level and subject matter taught, years of 
experience, and age. Although this is the most thorough and syste- 
matic work to date, it is not really a measure of a philosophy of 
education but rather of a psychological dimension of attitude to- 
ward pupils and those aspects of teacher behavior which relate to 
inter-personal relationships, teacher’s organization (poised-excit- 
able, disorganized-systematic, ete.) and mode of presentation (dull- 
stimulating, sterotyped-original). 

Kerlinger (1961) has developed scales which measure educa- 
tional “progressivism” and “traditionalism,” but not the identity 
and logical consistency of a person’s ontological, epistemological 
and axiological conceptions, His Likert-type scales grew out of 
“Q” sort beginnings. Each scale consists of 10 “progressive” and 
10 “traditional” items, He reports: “This study must still be con- 
sidered exploratory. . .. More work needs to be done with different 
items and different samples and longer scales” (Kerlinger, 1961, 
p. 284). 


Procedure 


With its one hundred items devised with the help of scholars of 
philosophy the GNC was easily the most extensive and authorita- 
tive source of items and thus the obvious resource with which to 
begin. The GNC was transformed from a Q-sort instrument into 
an ordinal attitude scale and administered to undergraduate and 
graduate classes in the philosophy of education at the University 
of Florida. Upon item analysis, 40 of these 100 GNC items yielded 
significant discriminatory power to measure the degree and con- 
sistency to which a person’s conception of education is experimental 
or rationalistic in the three areas of ontology, epistemology and 
axiology. 

These forty items were then successively combined with those 
items from the work of Sayers, Ryans, Kerlinger and Oliver (1953) 
which were felt to be “philosophic.” In addition, a set of epistemo- 
logical items was included from the work by a faculty committee 
charged with the development of a list of concepts which were 
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thought to be important for graduates of the college to hold. Fur- 
ther item analyses were made on the new instruments thus evolved. 

These successive item analyses yielded a final pool of fifty items 
which had, over the several test administrations with graduate and 
undergraduate University of Florida classes, maintained statisti- 
cally significant discriminatory power. The task then shifted to 
selecting from these fifty items a short schedule of items which 
would reliably and validly measure groups on the continuum of a 
conceptual philosophy of education that ranged from most ration- 
alistic to most experimentalistic. 


Sample for Final Analysis 


Table 1 presents the number and arrangement of the groups 
whose test scores were used to provide the data for the final item 
analysis and reliability study. Two administrations of the test 
were given, a month apart, since reliability was to be of the test- 


TABLE 1 


Number and Arrangement of Groups Used in Item Analysis 
and Reliability Test Administrations 


TEST ADMINISTRATION 


TIME I TIME II 
GROUP N GROUP N 

A; Total Group 292 Ay Total Group 292 
Graduates 78 Graduates 78 
Seniors 117 Seniors 117 
Sophomores 97 Sophomores 97 
B Total Group 62 Cc Total Group 58 
Graduates 14 Graduates 19 
Seniors 23 Seniors 28 
Sophomores 25 Sophomores 11 
TIME I TOTAL 354 TIME II TOTAL 350 


retest model. It was anticipated that this design would provide 
three distinct groups: group A being those who were present for 
both administrations of the test; group B those who were present 
for the first administration but not the second; and group C those 
who were present for the second administration but not the first. As 
it turned out, group A yielded a sample of 292 persons, while groups 
B and C numbered 62 and 58 students, respectively. 

The test results from group A (Time 1) were used to make the 
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item analysis, and these same test results taken in conjunction with 
the Time II test administration with Group A supplied the data 
upon which the test-retest reliability estimate was made. In order 
to cross validate the results of the item analysis performed with 
the group A test scores, groups B and © were combined into one 
group of 120 subjects and the analysis was repeated. No other use 
had been made of the results from group B and C, thereby making 
them a legitimate and time-saving second sampling suitable for the 
purpose of cross validating the initial item analysis. 

The problem of empirical validity was approached through se- 
lecting a sample that would be representative of populations which 
had had different amounts of exposure to the University of Flori- 
da’s program of teacher education. With this criterion in mind, 
three sub-groups were established: sophomores, seniors, and gradu- 
ate students, The sophomore group consisted of those taking their 
first courses in education and were, for the most part, those en- 
rolled in courses in human growth and development. The senior 
group were those who were just finishing their undergraduate pro- 
grams, but who had not as yet begun their teaching internships. 
Finally, the graduate group comprised students who had entered a 
graduate program directly from undergraduate study as well as 
many who had returned to the University after an indeterminate 
number of years in teaching or other fields, 

It was assumed that the University of Florida College of Edu- 
cation faculty generally subscribes to an experimental conception 
(as expressed by Dewey, Mead, Kilpatrick, Bode, Kelley, Stanley 
and the like), Therefore, it was further assumed that a division of 
the student body into sophomore, senior and graduate level ex- 
posure to this position would yield a criterion population with the 
graduates tending to express the more experimental and the sopho- 
mores the more rationalistic philosophy of education, 


Data Analysis 


Two measures of item discriminatory power were used, each of 
them based on the upper and lower 27 per cent of the distribution. 
The t-score discrimination index i 
(1957) and the PHI index is t 
(1954). The t-score index is suita 
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which items are scored on Likert-type scales, while the PHI index 
requires the more traditional correct-incorrect type of scoring. 

Subjects scored items on a four point Likert-type scale, the cate- 
gory ratings being the following: Agree Strongly, Agree, Disagree, 
Disagree Strongly. Responses on the non-experimental items were 
scored 1, 2, 3, and 4, respectively. For the experimental items the 
scoring weights were simply reversed. Computation of the t-score 
index therefore was based on fully weighted scores, while computa- 
tion of the PHI index required that scale values 1 and 2 be com- 
pressed into the single value 0, and scale values 3 and 4 into the 
value 1. 

Item difficulty level (D) was computed on the basis of the com- 
pressed correct-incorrect (in this case, experimental-nonexperimen- 
tal) scoring, using the total number of subjects. 

In selecting items for inclusion in the final form of the instru- 
ment, consideration was given to the three indices reported. Each 
index, of course, was interpreted in the light of its own criterion. 
In the case of the t-score index, the suggested procedure is to select 
the required number of items in the order of the magnitude of t, 
taking a t of 1.75 as a minimally acceptable lower limit (Edwards 
1957, p. 152). Following Guilford’s procedures for estimating the 
significance of PHI with samples of different sizes, the required 
magnitude of indices in this study were: for the initial analysis, .11 
at the .05 level of significance, and .14 at the .01 level; for the vali- 
dation analysis, .18 at the .05 level, and .23 at the .01 level of 
significance (Guilford, 1954, p. 432). With respect to item difficulty 
it was decided that for the purposes of this study, no particular con- 
cern would be expressed for any item remaining within a range 
bounded by .20 and .80. 


Results 


For the most part, these criteria were met in the final item analy- 
sis and sustained in the cross validation analysis for the twenty- 
five items selected for inclusion in the final form of the instrument. 

‘As a result of the final item analysis, the following twenty-five 
items were selected as the most usable in a short test which would 
measure a subject’s predisposition to express & philosophy of edu- 
cation that could be termed experimentalistic. 

1. In this period of rapid change, it is highly important that edu- 
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cation be charged with the task of preserving intact the long es- 
tablished and enduring educational aims and social objectives. 

2. The true view of education is so arranging learning that the 
child gradually builds up a storehouse of knowledge that he can use 
in the future. 

8. In assessing what man knows, there are no absolutes, only 
tentative conclusions based on the current accumulation of hu- 
man experiences, 

4. Required reading of literary works, even though it may bring 
an unfavorable attitude toward literature, is necessary in a sound 
educational program. 

5. To learn means to devise a way of acting in a situation for 
which old ways are inadequate. 

6. In the interest of social stability, the youth of this generation 
must be brought into conformity with the enduring beliefs and in- 
stitutions of our national heritage. 

7, Learning is a process of mastering objective knowledge and 
developing skills by drill, trial and error, memorization, and logical 
deduction. 

8. The teacher must indoctrinate her students with correct moral 
principles in order to bring about their healthy moral develop- 
ment, 

9. Moral education is the continuous criticism and reconstruc- 
tion of ideals and values. 

10. The traditional moral standards of our culture should not 
Just be accepted; they should be examined and tested in solving the 
present problems of students. 

11. The backbone of the school curriculum is subject matter; 
activities are useful mainly to facilitate the learning of subject 
matter, 

12, A teacher may properly teach that some laws are unchanging 
and certain in their essential nature, 

13. Moral learning is experimental; the child should be taught to 
test alternatives before accepting any of them. 

14, Minimum standards of achievement, in the form of require- 
ments to be met equally by all students, must be demanded at 
every level of education, 

15. Existing knowledge is tentative and is subject to revision in 
the light of new facts. 


a 
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16. A knowledge of history is worthwhile in itself because it em- 
braces the accumulated wisdom of our ancestors. 

17. An activity to be educationally valuable should train rea- 


soning and memory in general. 

18. The teacher is a channel of communication, transmitting 
knowledge from those who know to those who do not know. 

19. The best preparation for the future is a thorough knowledge 
of the past. 

20. The curriculum should contain an orderly arrangement of 
subjects that represent the best of our cultural heritage. 

21. Child life is not a period of preparation, but has its own in- 
herent value. 

92. The aim of instruction is mastery of knowledge. 

23. There is no reality beyond that knowable through human 


experience. 
24, Learning is essentially a process of increasing one’s 8 
lds of knowledge. 


information about the various fie’ 
25. Only that should be accepted as true which meets the test of 


experience. 


tore of 


Table 2 presents the item analysis results for both the final 
analysis and the cross validation analysis. In addition, each item is 
identified as either an E (experimental statement) or an R (ra- 
tionalistic statement), classification dependent solely on the item’s 
content value. Support for the validity of these classifications rests, 
of course, on the fact that all but items No. 3, No. 15, No. 21, No. 23 
and No. 25, were identified as experimental or rationalistic in previ- 
ous instruments or judgments. These five items were from the origi- 
nal set developed by the faculty committee. 

When subjected to the cross-validation analysis, one item, No. 
25, fell below both the score and the PHI criteria for admissabil- 
ity and is therefore not recommended for future use. Ttem No. 15 
did not reach significance On the PHI index. Also, this item, to- 
gether with item Nos. 1 and 3, had a difficulty level which was out- 
side the criterial range. It is recommended, however, that these 
three items be retained. The deviation of item No. 1 from the cri- 
terion of .80 was slight and occurred only in the initial analysis. 
culty level, still had significant dis- 


Item No. 3, despite a high diffi ATIA 
criminatory power. It is hoped that further administrations of the 
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TABLE 2 
Item Analysis Results Reported in Terms of Discrimination 
and Difficulty Level Indices 
Cross Validation 
Content Final Analysis Analysis 
Classifi- 
cation: (N = 292) (N = 120) 
Rationalistic Discrimination Discrimination 
or Difficulty Diffculty 


Item Item* Experi- 
Number Source mental tscore PHI D  tscore PHI D 
8: 


r GNC R 6.15 .35 .82 6.22 .43 .80 
2. K R 5.98 49 .42 4.58 60 .49 
3. UF E 5.42 19 90 5.16 18.81 
4, GNC R 7.85 57 58 5.79 .80 .58 
5. S E 2.53 .25 -63 3.29 .20 .58 
6, GNC R 4.35 -40 -66 6.61 .60 .69 
is GNC R 8.34 58 51 4.82 -57 .50 
8. GNC R 8.24 -49 72 5.88 -67 68 
9, S E 3.15 .28 70 2.20 .18 .66 
10. K E 5.16 38 -70 4.55 -i y TY bs} 
11 K R 6.83 -49 .55 4.12 .40 .57 
12. GNC R 6.14 AT .39 6.04 47 42 
13. K E 5.98 44 OF) 2.78 .43 «39 
14, GNC R 8.31 64 -60 3.12 -50 54 
15. UF E 4.68 -03 -98 3.36 E10. 4.97 
16. GNC R 6.10 -28 -70 5.16 .60 .40 
17. R R 7.81 -56 -40 4.61 43 37 
18. GNC R 9.20 -67 -39 3.26 .53 41 
19. GNC R 2.53 44 -60 4.20 -43 57 
20. K R 4.21 31 37 4.22 -40 46 
21, UF E 7.96 -40 tein (3.98 -40 69 
22. GNC R 8.24 51 AO. STO 47 70 
23. UF E 5.29 -40 43 2.47 -20 34 
24, K R 9.55 +75 44 4.69 -50 47 
254 UF: E E Ao s y AR 10.45 

Significance of PHI: for, N = 292: p < .05, «all; p < .01, 14. 

for N = 120: :205, .18; 
* GNC—Gowin, Newsome, and EER Soh si? < OL 28. 
K—Kerlinger 
UF—Faculty Committee, College of Education, University of Florida 
S—Sayers 


R—Ryans 
b Not recommended for inclusion in future use of the instrument, 
instrument with groups less experimental than those at the College 
of Education, University of Florida, will reduce the difficulty level 
of Item No. 15 and possibly also increase its discriminatory power. 
In the meantime, it serves the purpose of keeping the number of E 
items in the instrument from falling below a third of the total 
number of items. Even if future administrations of the instrument 
do not vindicate the inclusion of item No. 15 on the basis of in- 
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creased discriminatory power, it should still be kept in order to 
hold the type of items in balance and possibly avoid response 
“gets.” Under these circumstances, the item should, however, be dis- 
regarded in computing a subject’s score on the instrument. 


TABLE 3 
Means, Standard Deviations, and Reliability r’s for Final Analysis 


TEST ADMINISTRATION 


TIME I TIME II 

$ Group N MEAN SD. Group N MEAN 8D. 

A; Total Group 292 67.49 s02 An TotalGroup 120 67.06" 8.19 

D Graduates (78 71.82% 0.22 Graduates (78) 722s 9:10 
Seniors 117 67.56" 6.38 Seniors 117 67.44* 6.78 


Sophomores 97 62.77* 6.32 


As Table 3 shows, the test-retest reliability correlation coefficient 
for the total group of 292 students was .82. When the subgroups 
composing the total sample were examined, comparable correlations 
(.83 and .80) were found for the graduate students and the sopho- 
mores. The seniors, however, had a reliability correlation of only 
72. Table 3 also reports the subgroup mean scores, together with 


their standard deviations for both the Time I and Time II test 


administrations. The instrument had been scored in such a way 
] philosophy of the sub- 


that the more experimental the educationa’ 
ject the higher the score he would receive. The graduate students 
were significantly more experimentalistic than the sophomores. ‘hat 


this could be taken as partial confirmation of the validity of the 


instrument resided in the fact, already mentioned, that this should 


be the expected distribution in the light of the type of professional 


development offered at the University of Florida’s College of Edu- 


cation. 
In order to establish the stability of these differences found 


among the graduate, senior and sophomore groups, & replication was 
populations. The results 


performed with another sample from these f : 
are found in Table 4. In this table, since all the seniors available 
for testing were currently interning, this group 1s referred to as 


interns. : 
A change was made in the method of scoring the instrument. The 


82 


83 
72 
80 


A Sophomores 97 63.70* 6.73 
*p < 01 
s Maximum possible score = 100, minimum = 25. 
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TABLE 4 


Test Validity Reported on Basis of Replication of 
Significant Sub-Group Mean Differences 


Group N Mean 8.D. 
1. Graduates* 42 63.50* 11.83 
2. Interns 211 57.36* 10.66 
3. Sophomores* 50 46 .98* 7.79 


* Different from other two groups, p < .001 

* Point-biserial r between total score and group membership was .63 @ < .01) 
scoring scale was expanded from a four-point scoring band of Agree 
Strongly, Agree, Disagree and Disagree Strongly to one that in- 
cluded a middle category of No Opinion. The weighting of re- 
sponses was also changed from a 1 through 4 range to a 0 through 
4 range. As Table 4 indicates, the subgroup means were again found 
to differ significantly and in the direction expected. Validity was 
also demonstrated by finding that a point biserial correlation of 
68 existed between total test scores and a student being in either 
the graduate (most experimental) or sophomore (least experimen- 
tal) group. If, as it seems reasonable to assume, the student body at 
the University of Florida College of Education becomes more ex- 
perimental and less rationalistic in stated philosophic outlook the 
longer a student has been in attendance, then the instrument pre- 
sented here appears to be a valid, as well as short and reliable, 
measure of that dimension of an individual’s philosophy of edu- 
cation. 

In constructing this instrument it was discovered that the popu- 
lation sampled was skewed in the direction of the experimentalist 
position. Despite the paucity of subjects Possessing a rationalist 
philosophy of education, the items were able to yield satisfactory 
discriminatory power. It Seems reasonable to conjecture that both 
item discrimination and test validity coefficients would be strength- 
ened if the test were now to be administered to a criterion sample 
which included a larger representation of rationalist subjects. Veri- 
fication of this expectation would appear to be the next logical step 
in any further research on this instrument. 
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SOME DIMENSIONS OF MEANING OF THE 
CONCEPT TELEVISED INSTRUCTION? 


FRED W. OHNMACHT? 
University of Maine 


Tym flexibility, efficiency and economy of the Semantic Differen- 
tial has led to its widespread application in the measurement of at- 
titudes towards a variety of stimuli. Recent studies by Greenberg 
(1963), Janes (1964), Kraus (1962) and Westley and Jacobson 
(1963) are among the many which testify to the popularity of the 
technique. Typically, investigators have selected scales which rep- 
resent the Evaluative, Potency, and Activity factors as identified 
by the work of Osgood (1957) and his associates. Responses to 
the scales are compared individually and values for the three fac- 
tors are derived through the summation of the scales representing 
each one. The summed evaluative scales are thought to represent 
attitude toward the stimulus rated. 

Now the fact that Evaluation, Potency, and Activity factors 
typically turn up when matrices of intercorrelations of scales are 
factored is a most interesting finding. However, this in no way sug- 
gests that scales loading on these factors when they are first sum- 
med across a large number of concepts are appropriately used for 
the estimation of the aforementioned factors when specific concepts 
are the point at issue. This point does not appear to be sufficiently 
appreciated by a number of investigators. 

Osgood (1957) has pointed out that when responses to a Se- 
mantic Differential for a single concept are factor analyzed a good 
deal of concept-scale interaction may take place. A recent study by 

1 The research reported here was supported by the College of Education 
Team Teaching Project which is funded by the Ford Foundation. 

2 Now at the University of Georgia. 
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Husek and Wittrock (1962) demonstrated the same point when it 
was found that scales utilized as estimators of the Potency and Ac- 
tivity factors aligned themselyes with other evaluative scales when 
the concept “School Teachers” was rated by means of a form of the 
Semantic Differential. Hartman (1963), in his recent review, sug- 
gests that users of the Semantic Differential would be well advised 
to perform their own factor analyses prior to using the technique. 
This suggests that investigators should be wary of measuring the 
meaning of specific concepts with scales identified through analyses 
which have collapsed data across concepts. 


Purpose 


Tepresentation for the Evaluative, Activity, and Potency factors 
if they characterized dimensions underlying the data. 


( 1960). The resulting principal-component structure was subjected 
to varimax rotation (Kaiser, 1958). The scale loadings on the re- 


ts 
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Findings 

Seven principal components extracted from the 26 26 matrix 
of scale intercorrelations had latent roots greater than unity. These 
seven components were rotated to the varimax criterion. Table 1 
presents the loadings of the 26 scales on the seven rotated reference 
axes. In order to simplify the table, only loadings of .40 or higher 
are shown. 

An examination of the hyperplane counts (+.15) for the seven 
rotated factors discloses a reasonably clear simple structure. Factor 
I, however, appears to be a general factor with but five scales hav- 
ing a loading within .00 + .15. 


Factor I-General Evaluation 


This factor is clearly a general evaluative factor which accounts 
for approximately twice as much variance as any other factor. 
Scales such as good-bad (.76), meaningful-meaningless (.81) and 
positive-negative (.57) mark the factor as evaluative in nature. 
The general factor subsumes several “modes” of evaluation since 
the scales colorful-colorless (.69) and interesting-boring (.70) have 
their major loadings on this factor. These scales have been identi- 
fied by Osgood (1957) as loading on a Receptivity factor which 
represents a mode of evaluation independent of the more commonly 
found Evaluation factor. Perhaps the most striking loading on Fac- 
tor I is that of the weak-strong scale (—.78). This scale is usually 
thought of as a marker for the Potency factor. Osgood (1957) has 
suggested that the greater emotionality involved in a concept, the 
greater the tendency for scales representing evaluation and other 
dimensions found in more general analyses to converge. The rota- 
tional affects noted with respect to Factor I suggests that this is 
the case for the concept Televised Instruction and reinforces the 
notion that the semantic differential must be adapted to the re- 
quirements of a research problem, preferably by a preliminary 
analysis of concept-scale interaction when attitudes toward partic- 
ular stimuli are at issue. 


Factor II-Relation of TV Instruction to Pupil 


Five scales have a rotated loading in excess of 40 on this di- 
mension. The scales warm-cool and passive-active have their ma- 
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TABLE 1 


Rotated Factor Loadings of Scores on Twenty-Siz Semantic Differential 
Scales (Concept: Televised Instruction) 


Factor 
Scale I II m IV N. VI VI hs 
2. heavy-light -72 59 
3. unfair-fair -58 56 
4. small-large 65 -57 
5, fast-slow 74 “70 
6. unimportant- 
important —.60 64 
7. colorful-colorless .69 58 
8. interesting-boring 70 -14 
9. mild-severe -83 ‘2 
10. unpleasant- 
pleasant —.47 .58 -76 
11. positive-negative .57 +56 
12, weak-strong —.78 74 
13. complex-simple -86 «79 
14. changeable-stable .57 .40 +59 
15, disreputable- 
reputable -78 -67 
16. meaningful- 
meaningless .81 «76 
17. good-bad «76 -72 
18. impersonal- 
personal 56 —.42 -59 
19. masculine-feminine .53 .52 74 
20. passive-active -68 +72 
21. deep-shallow AL 64 +74 
22. successful- 
unsuccessful -66 59 
23, warm-cool —.74 70 
24, excitable-calm AL 40 -58 
25, constrained-free 55 i 61 
26. sensitive-insensitive 40 —.62 58 
Hyperplane Count +.15 5 eli) iTS l 
% of Total Variance 20.6 9.5 10.5 5.9 Pe 70 59 
% of Common Variance 31.2 14.3 15.9 8.9 10.1 10.6 8.9 


meaning similar to one identi 
Activity. 


t 
w 
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Factor III-Personal Feeling about TV Instruction 


This factor is most clearly represented by the reputable-disreput- 
able (.78) scale. Other scales having loadings in excess of .40 are 
small-large (.65), unfair-fair (.58) and unpleasant-pleasant (.58). 
The interpretation of this factor does not seem as clear cut as for 
Factor I and IT. 


Factor IV-Complexity (Specific) 


Factor IV is highly specific to the complex-simple (.86) scale with 
the impersonal-personal (—.42) being the only other scale with a 
loading in excess of .40. The lack of other scales with reasonably 
high loadings on this factor make its identification tenuous at best. 


Factor V-Severity 


Factor V is characterized by the scales mild-severe (.83), 
deep-shallow (.64) and masculine-feminine (.53). These scales 
often are loaded on a Potency factor but in the present instance 
scales loading on Factor VII also suggests a Potency factor. It 
would appear that the Potency factor usually found has been bro- 
ken up in the present analysis into two separate factors with some 
scales (strong-weak and sensitive-insensitive for example) usu- 
ally found to load on the Potency factor rotating to the Evaluation 
and Activity dimensions. 


Factor VI-Pacing 


This factor is characterized by the scales fast-slow (.74), 
changeable-stable (.57) and yaluable-worthless (.49). The first 
two scales mentioned suggest that the Activity dimension has been 
separated into two independent parts (Factor II and VI). The val- 
uable-worthless scale also loads on the Evaluative factor and sug- 
gests that subjects conceive of televised instruction which is 
quickly paced and changeable as being “good.” f 


Factor VII -Dynamism 


This factor, along with Factor IV, accounts for the smallest 
amount of variance. The scales heavy-light, masculine-feminine, 
excitable-calm, and changeable-stable have loadings in excess of 
40 on this factor. It is interesting to note the position of the mascu- 
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line-feminine scale in the semantic space, especially with respect to 
Factors VII and V. On the one hand (Factor V), feminine tends to 
go with severe and shallow, while on the other hand (Factor VII), 
feminine goes with light, calm and stable. This suggests some am- 
bivalence in the allocation of the masculine-feminine scale in the 
semantic space. 


Application of Evaluative Scales 


Scales having a loading of .60 or higher on the General Evalua- 
tion Factor (Factor I) were used a number of times to assess col- 
lege students attitudes towards video taped televised lessons which 
they have just finished viewing. The seven scales thus used are 
scaled from one to seven and an attitude or evaluation score is ob- 
tained by summing the weights for the individual items to obtain 
an attitude score. In seven such applications the mean values ob- 
tained have ranged from 23.58 to 41.56, indicating that the college 
students involved do discriminate among various televised lessons 
in terms of the General Evaluative dimension. Total-score scale 
reliabilities for the seven administrations have ranged from .91 
to .96 in terms of Hoyt analysis of variance estimates. 


Discussion and Conclusions 


The findings of this study indicate that Evaluation is a major 
dimension of the meaning of the concept televised instruction. The 
presence of such a dimension is consistent with a growing body of 
work with the semantic differential. Factors Tepresenting the Ac- 
tivity and Potency dimensions are not as clearly delineated. In 
part, this may be a function of the pool of 26 scales used. Several 
scales usually associated with dimensions of meaning other than 
Evaluation had their major loadings on the Evaluation factor. This 
indicates that Concept-scale interaction occurs when the concept 
televised instruction is rated using the semantic differential. This 
is especially clear when it is observed that the scale weak-strong, 
which is a standard marker for the Potency factor, has a loading 
of —.78 on the Evaluative factor. 

In the present case, an investigator who chose three or four 
scales each to represent the Evaluation, Potency, and Activity fac- 
tors from the general analyses of Osgood would run the risk of 
dealing in numerology rather than establishing scores representing 
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the three dimensions of interest. The inclusion of the scale weak- 
strong to represent Potency would be basically in error and it is 
doubtful that it would be included as an estimate of Evaluation as 
the present study indicates it should be. The extent to which such 
concept-scale interaction occurs when other concepts are rated is 
typically unknown but is subject to empirical test. 

The present study provides a pool of items with known factorial 
composition for the study of attitudes towards televised instruc- 
tion. In addition this study reemphasizes the need for investigators 
to perform their own factor analyses when the nature of concept- 
scale interaction with respect to a given study are unknown. The 
semantic differential is not a definite set of items to be used with 
any and all concepts and must be adapted to the requirements of 
each research problem. 
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COMPARISON OF CRITERION CLUSTERS OBTAINED BY 
ANALYZING THE HOMOGENEITY OF A SET OF 
REGRESSION EQUATIONS AND THE MATRIX OF 
INTERCORRELATIONS 


JOHN P. CAMPBELL 
University of California, Berkeley 


Srrvations frequently occur in which there are available a fairly 
large number of variables, such as a set of criterion measures, and 
the researcher wishes to identify the dimensions of common vari- 
ance, Various methods of cluster or factor analysis (Harman, 1960; 
Tryon, 1964) are available for handling this situation but they all 
require that each individual be measured on an appreciable number 
of the variables. Unfortunately, this situation does not always exist 
in the real world and a complete covariation matrix cannot be ob- 
tained, An extreme example of such a situation would be when each 
individual is measured on only one variable or criterion. Under 
these conditions no indices of covariation could be obtained for 
pairs of variables and a cluster or factor analysis could not be per- 
formed. In hopes of offering an answer to the above dilemma the 
present investigation uses and evaluates a technique for clustering 
variables based on the similarity of their regression equations 
rather than their intercorrelations. Since it is not dependent on the 
intercorrelations between criteria, it thus might offer some hope for 
overcoming an all too prevalent constraint. 


Method 


Bottenberg and Cristal (1961) have presented a technique for 
hierarchically clustering a set of regression equations into opti- 
mally predictable groups such that a smaller number of regression 
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equations can be used to predict the individual criteria with mini- 
mal loss of predictive efficiency. A computer program for carrying 
out this procedure has been developed by Ward (1961). From ex- 
amination of expressions (1) and (2) below it can be seen that this 
technique accomplishes its purpose by grouping together homogene- 
ous or similar regression equations. For purposes of this investiga- 
tion it seemed reasonable to assume that if two criteria are highly 
intercorrelated then their individual regression equations, when a 
common set of predictors is used and when the multiple R is rea- 
sonably high, should tend to “look alike.” That is, the individual 
predictors should have similar correlations with the two criteria 
and their standard partial regression weights should be similar for 
the two equations. Criteria could then be clustered on the basis of 
the relative similarity of their regression equations. 

The Bottenberg and Cristal procedure begins by computing a 
weighted average of the individual multiple correlation coefficients 
using the following expression. 


È nps + > n.X2 — X?, 


a N a 


Where: g = the total number of criterion groups (including un- 
grouped individual criteria), 


nı = number of observations in criterion group i. 

R? = squared multiple correlation of predictor battery 
with criterion group i. 

8? = variance of criterion scores in group i. 

X? = mean of criterion scores in group i. 

N = total number of individuals in all criterion groups. 

X. = grand criterion mean, 

& = grand criterion variance, 


The two individual equations are then combined which reduce 
this over-all average R? the least, To do this, every possible pair of 
the original set of g equations is evaluated by means of the follow- 
ing expression, 
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Where: m = the number of criteria in the cluster. 
By = standardized regression weight for the jth predictor 
of criterion 7. 
Vay = corresponding validity coefficient for the jth pre- 
dictor of criterion h. 
s? and s2 = variance of the observations in the individual cri- 
terion groups. 
n; and ny = number of observations in the individual criterion 
groups. 
N = sum of the observations in the I criterion groups. 


The subscripts i and h both take on all possible values. In the 
case of evaluating pairs of equations the double summation in ex- 
pression (2) will be over four different values. om)? is thus an 
average quantity obtained by using all possible combinations of 
zero-order correlations and standardized partial regression coeffi- 
cients. The more similar the correlations and Beta weights for two 
criteria, the smaller will be the relative decrement in Rim. In 
other words, by means of expression (2) the total number of multi- 
ple correlation coefficients is reduced from g to g-1 and that com- 
bination is made which minimizes the difference between the 
average of g oRıg)”s and g — 1 oF isy’s. The highest average will al- 
ways be obtained when an individual equation is used for each 
criterion. Criteria are clustered one at a time, and this hierarchical 
grouping procedure is continued until all variables or criteria have 
been combined into one group and Rg)? is at a minimum, 

In order to make a one-to-one comparison between the multiple 
regression method and an analysis which utilizes a covariation 
matrix, the Holzinger-Harman (Fruchter, 1954) method of cluster 
analysis was modified so that variables were clustered in a hier- 
archical fashion and the variables within clusters were as homo- 
geneous as possible with regard to their intercorrelations. With this 
method the index of cluster homogeneity is the B-coefficient, or ratio 
of the average of the intercorrelations of the variables within a 
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cluster to the average of the correlations of the cluster variables 
with the variables outside the cluster. Variables were clustered one 
at a time by computing the B-coefficient for every possible combina- 
tion and making that combination which maximized the B-coeffi- 
cient or average of the B-coefficients at each stage. This procedure 
starts by clustering the two variables which have the highest inter- 
correlation. The second step involves either clustering a third vari- 
able with the initial pair or clustering two variables to form a 
second pair. The decision is dependent on which is larger, the B- 
coefficient for the three variable cluster or the average of the B- 
coefficients for the two pairs. Variables are clustered in this fashion, 
one at a time, until all variables are grouped into clusters. These 
clusters are then grouped to form larger clusters until all variables 
have been clustered into one group. Thus the number of stages cor- 
responds to the number of variables, and at each stage the most 
homogeneous grouping is achieved in terms of the variable inter- 
correlations. 


Subjects 


The individuals used in the study were 406 students in the Uni- 
versity of Minnesota’s Institute of Technology who had completed 
the first two years of the engineering curriculum, 


Predictors and Criteria 


Five predictors were included: (1) high school tank, (2) the 
Minnesota Scholastic Aptitude Test (MSAT), (3) the Mechanics 
of Expression score and (4) the Effectiveness of Expression score 
from the Cooperative English Test: Lower and Higher Levels, and 
(5) the Institute of Technology Mathematics Test (IT Math). 
Predictors two, three, and four are regularly given to Minnesota 
high school juniors as part of the university counseling bureau’s 
statewide testing program (Berdie, et al, 1959). The IT Math test 
is given to all high school seniors who apply for admittance to the 
Institute of Technology, 

The criteria consisted of twenty of the basic core courses taken 
by almost every engineering student during his freshman and soph- 
omore years. They are listed in Table 1, 


i 
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TABLE 1 


The Basic Set of Twenty Core Courses in the Engineering 
Curriculum That Were Used as Criteria 


1) Composition 1A (Comp 1) 10) Calculus I (Cale 1) 
2) Composition 2A (Comp 2) 11) Calculus IT (Cale 2) 
3) Composition 3A (Comp 3) 12) Calculus III (Cale 3) 
4) Literature 1A (Lit 1) 13) Calculus IV (Cale 4) 
5) Literature 2A (Lit 2) 14) General physics 11 (Phys 11) 
6) Literature 3A (Lit 3) 15) General physics 12 (Phys 12) 
7) Engineering graphics 14 16) General physics 13 (Phys 13) 

(EG 14) 17) General physics 14 (Phys 14) 
8) Engineering graphics 15 18) Intermediate physics 50 (Phys 50) 

(EG 15) 19) General chemistry 15 (Chem 15) 
9) Algebra, trigonometry, 20) General chemistry 16 (Chem 16) 

and analytic geometry 
(Algebra) 
Procedure 


To check the relative stability of the two solutions the sample 
was split into two groups (A and B) of 203 individuals each and 
the analysis carried out in identical fashion on both groups. For 
each subsample the regression equations for each of the criteria 
and the criterion intercorrelation matrix were computed. Both the 
multiple regression and the cluster analysis technique were then 
used to cluster hierarchially the individual criteria. At each stage, 
for both methods, the multiple correlation of the predictor battery 
with each total cluster score and the intercorrelations of the cluster 
scores were obtained. 


Results 


The criterion clusters obtained by each method at selected stages 
are shown in Tables 2 and 3 for subsamples A and B respectively. 
As can be seen from these two tables the two methods tend to give 
dissimilar results when applied to the same data. This lack of simi- 
larity can best be illustrated by an attempt to conceptualize and 
name the clusters at say stages 15 through 19. 

For the analysis of intercorrelations the task is not too difficult. 
All the English courses group together in an apparent verbal factor 
which includes both the ability to write clearly and correctly 
(comprehension) and to understand what has been read (litera- 
ture). By stage 15 this cluster is complete and it does not change in 
successive stages. The two engineering graphics courses also group 

+ 
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TABLE 2 


Comparative Results of Hierarchical Clustering of Criteria on the 


Basis of Both the Homogeneity of Regression Equations and 


the Criterion Intercorrelations 


Sample A 
Stage By regression equation By intercorrelation 
(1) All criteria treated individually 
(2) Comp 1 Comp 3 
Comp 2 Lit 3 
(.43) (48) 
(3) Comp 1 Phys 12 Comp 3 
Comp 2 Phys 13 Lit 3 
(.43) (.41) Comp 2 
(.49) 
(4) Comp 1 Phys 12 Comp 3 Comp 3 
Comp2 Phys 13 Lit 3 Lit 3 
(.48) (.41) (.48) Comp 2 
Lit 2 
(.49) 
(5) Comp1 Phys 12 Comp 3 Comp 3 
Comp2 Phys 13 Lit 3 Lit 3 
(.48) (.41) (.48) Comp 2 
Lit 2 
EG 14 Comp 1 
Chem 15 (.50) 
(.45) 
(10) Comp1 Phys 12 Comp 3 Comp 3 Chem 15 Phys 12 
Comp 2 Phys 13 Lit 3 Lit 3 Chem 16 Phys 13 
Lit 2 (41) (.48) Comp 2 (.38) Phys 11 
(.45) Lit 2 Algebra 
Comp 1 (.52) 
EG 14 Cale 3 Cale 2 Lit 1 
Chem 15 Phys 50 Phys 14 (.51) 
Phys 11 (.29) (.83) 
(.55) 
Cale 1 
Chem 16 
(.42) 
(15) Comp1 Phys 12 Comp 3 Chem 15 Phys 12 
Comp 2 Phys 13 Lit 3 Chem 16 Phys 13 
Lit 2 EG 44 Comp 2 (.38) Phys 11 
Lit 1 Chem 15 Lit 2 Algebra 
Comp 3 Phys 11 Comp 1 Phys 14 
Lit 3 (.53) Lit 1 (.50) 
(.51) (.51) 
Cale 3 Cale 2 EG 14 Cale 2 
Phys 50 Phys 14 EG 15 Cale 3 
Cale 1 Algebra (.87) Cale 1 
Chem 16 (,44) Dif-E 
(.39) (.39) 
» (16) Comp 1 Phys 12 Cale 2 Comp 3 Chem 15 Phys 12 
Comp 2 Phys 13 Phys 14 Lit 3 Chem 16 Phys 13 
Lit 2 EG 14 Algebra Comp 2 (.88) Phys 11 
Lit 1 Chem 15 Cale 3 Lit 2 Algebra 


i a. 
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Sample A 
Stage By regression equation By intercorrelation 
Comp 3 Phys 11 Phys 50 Comp 1 Phys 14 
Lit 3 (.53) Cale 1 Lit 1 (.50) 
(.51) Chem 16 (.51) 
(.43) 


EG 14 Cale 2 
EG 15 Cale 3 
(.87) Cale 1 


Diff-E 
Phys 50 
(.37) 

(17) Comp 1 Phys 12 Cale 2 Comp 3 Chem 15 Phys 12 
Comp 2 Phys 13 Phys 14 Lit 3 Chem 16 Phys 13 
Lit 2 EG 14 Algebra Comp 2 (.38) Phys 11 
Lit 1 Chem15 Cale 3 Lit 2 Algebra 
Comp 3 Phys 11 Phys 50 Comp 1 $ Phys 14 
Lit 3 EG 15 Cale 1 Lit 1 Cale 2 
(.51) (.51) Chem 16 (.51) Cale 3 

(.43) Cale 1 
EG 14 Diff-E 
EG 15 Phys 50 
(.37) (.47) 


(18) Comp1 Phys 12 Cale 2 Comp 3 Chem 15 
Comp 2 Phys 13 Phys 14 Lit 3 Chem 16 

2 EG 14 Algebra Comp 2 Phys 12 

1 Chem 15 Cale 3 Lit 2 Phys 13 
Comp 3 Phys 11 Phys 50 Comp 1 Phys 11 
3 EG 15 Cale 1 Lit 1 Algebra 
Difi-E (.51) Chem 16 (.51) Phys 14 
(.47) (.48) Cale 2 
EG 14 Cale 3 

EG 15 Cale 1 


(.37) Diff-E 
Phys 50 
(.48) 
(19) All English All non-English All English All non-English 
courses + courses except courses courses 
Diff-E Diff-E (.51) (.49) 
(.47) (.49) 
(20) All courses combined All courses combined 
(.52) (.52) 


Note—Only those criteria which were clustered at Stages 1-5, 10, and 15-20 are shown. The 
multiple correlations of the predictor battery with the criterion cluster score is given 
in parentheses. 


together. At stage 15 all the calculus courses and the courses in 
differential equations group together, but the course in algebra 
clusters with the physics courses. On the basis of course content the 
calculus courses and differential equations would seem to represent 
a higher mathematics factor that requires a much more abstract 


* 
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TABLE 3 


Comparative Results of Hierarchical Clustering of Criteria on the 


Basis of Both the Homogeneity of Regression Equations and 


the Criterion Intercorrelations 
Sample B 

Stage By regression equation By intercorrelation 

(1) All criteria treated individually 

(2) Comp 2 Comp 2 
Lit 1 Lit 2 
(.48) (43) 

(3) Comp 2 Cale 3 Comp2 EG 14 
Lit 1 Phys 50 lit 2 EG 15 
(.43) (.40) (.43) (.36) 

(4) Comp2 Cale 3 Comp 3 Comp 2 G 14 
Lit 1  Phys50 Lit 2 Lit 2 EG 15 
(.43) (.40) (.48) pa. 1 (.86) 

(5) Comp2 Cale3 Comp3 Comp2 EG 14 
Lit 1 Phys50 Lit 2 Lit 2 EG 15 
(.48) (.40) (.48) Comp1 (.36) 

Lit 1 
Cale 2 (.48) 
Phys 14 
(46) 

(10) Comp 2 Cale 3 Comp 3 Comp2 EG 14 
Lit 1 Phys50 Lit 2 Lit 2 EG 15 
Comp 1 Cale 2 = (.48) Comp (.36) 
(.47) Phys 14 Lit 1 

(.42) Comp3 Cale 3 
EG 14 Phys 11 Lit 3 Dif-E 
Phys 12 Algebra Chem 15 (53) (.37) 
(.46) Cale 1  (.64) 

(.53) 

(15) Comp2 Cale3 EG 14 Comp2 EG 14 
Lit 1  Phys50 Phys 12 Lit 2 EG 15 
Compi Cale2 Phys 11 Comp 1 (.36) 
Lit 3 Physi4 Chem15 Lit 1 
Comp3 = Diff-E (.54) Comp 3 
Lit 2 (.43) Lit 3 
(.53) (.53) 

Algebra Cale 3 Chem 15 
Cale 1 Dif-E Chem 16 
Chem 16 Cale 1 (.51) 
(.57) Cale 2 

(.46) 

(16) Comp2 Cale3 EG 14 Comp2 EG 14 
Lit 2 Phys 50 Phys 12 Lit 2 EG 15 
Comp1 Calc 2 Phys 11 Comp1 (.36) 

Lit 3 Physi4 Chem 15 Lit 1 

Comp3 Di-E (.54) Comp 3 

Lit 2 (.43) Lit 3 

(23) ‘ (.53) 

Algebra EG 15 Cale 3 Chem 15 
Cale 1 Phys 13 Diff-E Chem 16 
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TABLE 3—Continued 


Sample B 
Stage By regression equation By intercorrelation 
Chem 16 (.44) Cale 1 Phys 14 
(.57) Cale 2 Phys 50 


(.46) (.49) 
(17) Comp2 Cale3 EG 14 Comp2 EG 14 Phys 12 
Lit 2 Phys50 Phys 12 Lit 2 EG 15 Phys 13 


Comp 1 Calc 2 Phys 11 Comp 1 (.36) Phys 11 
Lit 3  Physl4 Chem15 Lit 1 Algebra 
Comp 3 Diff-E Algebra Comp 3 Cale 3 
Lit 2 (.43) Cale 1 Lit 3 Diff-E 
(.53) Chem 16 = (.53) Cale 1 

(.59) Chem 15 Cale 2 
EG 15 Chem 16 (.53) 
Phys 13 Phys 14 
(.44) Phys 50 

(.49) 


(18) Comp 2 Cale 3 EG 14 Comp2 EG 14 Phys 12 
Lit 2 Phys 50 Phys 12 Lit 2 EG 15 Phys 13 


Comp 1 Cale 2 Phys 11 Compi (.36) Phys 11 
Lit 3 Physl4 Chem15 Lit 1 Algebra 
Comp 3_—sCODiff-E Algebra Comp 3 Cale 3 
Lit 2 (.48) Cale 1 Lit 3 Diff-E 
(.53) Chem 16 (.53) Cale 1 
EG 15 Calc 2 
Phys 13 Chem 15 
(.57) Chem 16 
Phys 14 
Phys 50 
(.53) 
(19) All English All non-English All English All non-English 
courses courses = courses courses 
(.53) (.54) (.53) (.54) 
(20) All courses combined All courses combined 
(.59) (.59) 


Note—Only those criteria which were clustered at Stages 1-5, 10, and 15-20 are shown. The 
multiple correlations of the predictor battery with the criterion cluster score is given 
in parentheses. 

and qualitatively different kind of problem solving from that in the 

course in algebra. It is not too surprising that this course groups 

with the first three physics courses, since many of the skills required 
to solve introductory physics problems are algebraic in nature. The 
two chemistry courses, which also group together at stage 15, remain 
as a discrete cluster until they group with the general mathematic’s- 
science factor at a later stage. The‘only apparent ambiguity in 
these results is the behavior of the two more advanced physics 
courses. In subsample B they form a,separate cluster and then 
group with the chemistry courses. However, in subsample A, phys- 


4 


TT 
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ies 14 clusters with algebra and introductory physics while physics 
50 clusters with the higher mathematics courses. Again however, 
the content of these last two physics courses is qualitatively differ- 
ent (more advanced topics) from the more basic introductory 
courses taken during the first year of the engineering curriculum. 

The behavior of these clusters across successive stages also makes 
reasonable sense. The higher mathematic’s factor merges with the 
physics factor and then with the chemistry factor to form a general 
mathematic’s-science factor. The spatial relations factor remains 
distinct until at stage 19 it is forced to choose between the verbal 
factor and the mathematic’s-science factor and as would be ex- 
pected, merges with the latter. Again, the only exception to this 
conceptualization is the clustering behavior of physics 14 and phys- 
ies 50. poa 

Tt is much more difficult to attach a name and conceptual descrip- 
tion to the clusters obtained by grouping criteria with homogeneous 
regression equations. For example, at stage 15 for subsample A, the 
chemistry courses are split between two clusters and so are the 
mathematical courses. Although the first cluster contains all the 
English courses, the second cluster includes the first course in 
chemistry and engineering graphics and the first three courses in 
physics; the third cluster contains algebra, physics 14, and calculus 
2; and the fourth cluster is made up of the last course in physics, 
chemistry, and calculus, plus the beginning course in calculus. It is 
almost impossible to distinguish among these last three clusters in 
terms of criterion content. The situation does not improve much 
across successive stages. It is made even more difficult when the 
course in differential equations clusters with the English courses at 
stage 18! 

In subsample B, at stages 15 and 16, the situation is not any 
clearer than in subsample A. At stage 18 the picture is somewhat 
better than it was in subsample A, since differential equations does 
not cluster with the English courses and the advanced mathematic 
and physics group together in what might possibly be called an ab- 
atract quantitative reasoning factor. However, the third cluster re- 
mains very difficult to name or define. 

Examination of Tables 2 and 3 also indi¢ates differing degrees of 
stability across subsamples for the clusters obtained by the two 
methods. For the intercorrelation method the only discrepancy 
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across subsamples is the clustering of physics 14 and 50 with the 
chemistry courses in subsample B, but with two different clusters in 
subsample A. By contrast, there is a much greater number of dis- 
crepancies between subsamples for the regression analysis. 

As might be expected the average intercorrelation of the total 
cluster scores obtained by grouping Tegression equations tends to 
be somewhat higher than that for the intercorrelation method. 
Table 4 shows this comparison for stage 18. At this stage in the 
clustering process all 20 of the criteria have been grouped by each 
method, and the number of criterion groups are the same, 


TABLE 4 
Intercorrelations of Cluster Scores at Stage 18 for Both 
Methods and for Both Subsamples 
EE RIE a SN ee 
Cluster Score Intercorrelations 
Group & Regression Analysis Correlation Analysis 
1 2 8 1 2 8 
A .49 .63 -50 34 
73 48 
F = 638 F= 44 
B -36 41 21 42 
78 48 
F = .55 f= .37 


“The average of each aot of cluster score intercorrelations was computed by means of Fisher's 
Z transformation, 
Discussion and Conclusions 

The two methods apparently do not give similar results, at least 
within the limitations of the present study. Although there is no 
objective decision rule that can be used to support this contention, 
it seems obvious that the clusters obtained at stages 15, 16, 17, for 
example, would be conceptualized differently depending on which 
_ method was used. Furthermore, the clusters obtained from the 
analysis of intercorrelations make considerably more “sense” in 
terms of apparent criterion content. The results from this method 
also appear to be more stable across the two subsamples. 

The above situation is somewhat analogous to the problem of 
estimating the factor loadings of a set of “outside” variables from 
the factor matrix obtained on another set of variables. Techniques 
for doing this (Dwyer, 1937) also required the initial correlation 
f each outside variable with every variable in the matrix (in this 
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study, the correlation of each criterion with every predictor). How- 
ever, this type of procedure is most applicable to the situation 
where the original set of variables is large and the number of out- 
side variables is small. The more usual situation in applied predic- 
tion is the reverse, or the situation embodied in this paper. The 
limiting factor in both situations is the degree to which the original 
set of variables (predictors in this case) account for the common 
variance in the outside variables. Apparently predictors which 
yield multiple correlations of this magnitude, which seem to be 
typical for thé usual Teal world situation, do not account for enough 
of the common variance to make a clustering of regression equa- 
tions trustworthy as a substitute for an unobtainable covariation 
matrix, ‘ 

Independent of whether or not an index of regression equation 
homogeneity can be substituted in a relative manner for an inter- 
correlation, is the question of which set of clusters in the hierarchi- 
cal solution to use for purposes of obtaining cluster scores or making 
statements about criterion dimension. Tryon (1964) makes the 
point that the number of clusters or factors extracted from a set of 
“real” intercorrelations is somewhat arbitrary. The number cannot 
be decided on the basis of the “rank” of the matrix, since in order 
for all the residuals to become zero the number of factors must 
equal the number of variables. Factoring or clustering is merely 
continued until a relevant index such as the average squared re- 
sidual or average obtained communality reaches a certain value. 

If the aim of the researcher is to obtain clusters: of variables 
which define a reduced number of dimensions and yield useful clus- 
ter scores, it would seem that obtaining maximally homogeneous 
subsets in a hierarchical fashion would provide a considerable 
amount of information. In the present example it would certainly 
be possible to compute some index of factoring such as the average 
squared residual at each stage. It is also possible to examine the 
relative homogeneity of the clusters, the intercorrelations among 
cluster scores, and the predictability (multiple-R with predictor 
battery) of each cluster score, By utilizing the hierarchical proce- 
dure the changing pattern of all these indices can be noted. For 
many research purposes the reduction of the residuals to some 
specified level may not be the most important consideration, and a 
solution which optimized the independence and predictability of 
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simple sum cluster scores would be preferred. Thus for many applied 
problems a procedure which gives a picture of the changing pattern 
of these indices would seem to be very valuable. The investigator 
could then choose the pattern most advantageous to him. 


Summary 


The present study compared two methods of grouping or cluster- 
ing criteria into homogeneous subsets in hopes of finding a solution 
to the problem of incomplete covariation matrices. A technique 
developed by Bottenberg and Cristal was used to group criteria on 
the basis of the homogeneity or similarity of their regression equa- 
tions, given a common set of predictors, This technique was com- 
pared to the Holzinger-Harman method of cluster analyzing the in- 
tercorrelation matrix. The multiple regression technique does not 
depend on the intercorrelations of the criteria. Both methods pro- 
ceed by clustering one variable at a time in a stepwise fashion. The 
solutions obtained by the two methods were judged not to be com- 
parable; however, certain advantages of the hierarchical method of 
clustering were pointed out. 
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A FOURFOLD TABLE 
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Numerous and varied approaches have been presented in statis- 
tics textbooks and journal articles to explain what is meant by the 
correlation coefficient or the coefficient of determination. Among 
the most helpful expositions for the understanding of the meaning 
of predictive validity coefficients was that of Taylor and Russell 
(1939) who not only proposed the selection ratio but also furnished 
useful charts to indicate the proportions of examinees who would be 
successful, relative to (a) several cutting scores in the predictor 
variable, (b) different proportions of employees considered suc- 
cessful on the job, and (c) various values for a validity coefficient. 
Using the normal bivariate surface as a model, Guilford and 
Michael (1949) extended Taylor and Russell’s work and furnished 
charts that showed the relationship between the proportion of those 
selected who would be successful and the selection ratio (proportion 
selected) for specified proportions of individuals successful on the 
criterion and for different validity coefficients. A chapter presented 
by Ghiselli and Brown (1955, pp. 140-48) and another chapter 
prepared by Thorndike and Hagen (1960, pp. 169-72) have also 
furnished useful indices and charts that provide helpful insights in 
the interpretation of validity coefficients. Thorndike and Hagen’s 
presentation probably comes closest to the approach that will be 
developed in this paper. 
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Problem 


Despite the aid furnished by these sources, beginning students in 
courses in psychological statistics and measurement encounter con- 
siderable difficulty in understanding either the predictive validity 
coefficient or its squared value, the coefficient of determination. It is 
the purpose of the writer to present three simple indices as well as 
a chart to accompany one of them which in his teaching experience 
have appeared to aid students in grasping what is meant by a pre- 
dictive validity coefficient when the normal correlation surface is 
subdivided at the median in both the predictor and criterion vari- 
able. 

The underlying rationale involves essentially a comparison of the 
Proportion of individuals who are correctly placed on the criterion 
by the predictor with (a) the size of the validity coefficient r 
and (b) the value of the corresponding coefficient of determination 
7. A corollary to this rationale is a comparison of the propor- 
tion of individuals correctly placed in excess of a chance proportion 
of .50 (when r = 0) with the magnitude of both r and r2. 


Definitions of Terms 


In Figure 1 the fourfold table sets forth the definitions of the 
principal terms involved in the median dichotomization of the nor- 
mal correlation surface on the predictor and criterion dimensions. 
The symbols A, B, D and C stand for the numbers of individuals in 
the first, second, third, and fourth quadrants, respectively. The 
individuals in the first and third quadrants represented by the sum 
A + D constitute those who have been correctly placed or who may 
be designated as “hits.” Thus the frequency A stands for examinees 
who are above the median in both the predictor and criterion varj- 
ables, and D for those below the median on each of these two 
variables. The frequencies B and C in the second and fourth 
quadrants, respectively, designate individuals who have been in- 
correctly placed—the “misses,” since their relative position on the 
criterion variable is contrary to that on the predictor variable. In 
the normal bivariate model the sum A + D should be equal to 
B + C when the correlation is zero. As the correlation increases in 
positive value for a population of fixed size N, the frequency 
A + D becomes monotonically higher than the frequency B + C, 
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Test or Predictor Variable 


Above the 
Median 


Above A 
the False Negatives Valid Positives A+B 
(False Exclusion) (Valia Inclusion) Tania, 
Median "Misses" "Hits" 
success. 
a ( ) 
a 
9 
E 
H Below D c 
$ 
H the Valid Negatives False Positives 
(Valid Exclusion) {False Inclusion) C+D 
Median "Hits" "Misses" Alen 
(Failure) 
BYD are WoAtD CD 


Figure 1. Correlation surface subdivided by a medium test score which 
separates the population into low- and high-scoring groups of individuals and 
by a median criterion value which divides the same population into their suc- 
cessful and unsuccessful standing in job performance. 


which is becoming progressively smaller. (When the correlation is 
negative, the sum of B + C will exceed that of A + D). 

Each of the three terms which serve to designate the entries in 
the third, fourth, and fifth columns of Table 1 may be defined for r 
= 0 as follows: 


ARED 
ENA 

COED 
N 


= the proportion of individuals in the total population 
of size N who are correctly placed—the proportion 


of “hits”; 


— .5000 = 


the proportion of correctly placed in- 
dividuals in excess of the chance pro- 
portion of .50 which is associated with 
an r of zero; and 
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100(4+ 2) — .5000 
I= TRAMs ie the percentage of improvement in cor- 
k rect (valid) placements relative to the 


proportion of .50 for an r of zero. 


Similar definitions could be formed with B + C taking the place of 
A + D when r is negative. Although not presented, ratios of 
A+DtoB+CorofB+CtoA-+ D also furnish bases for in- 
terpretation of a correlation coefficient. 


TABLE 1 


Three Indices Furnishing Additional Information Regarding the Meaning of 
Predictive Validity for Various Values in r and r? When a Correlation 
Surface is Subdivided at the Median in Both the 
Predictor and Criterion Variables 


Corre- Coeffi- Proportion Proportion Percentage of Im- 
lation cient of Correct Correctly provement in Correct 
Coeffi- of (Valid) Placed in (Valid) Placements 
cient Determi- Placements Excess of Relative to Propor- 
nation (H) Chance Pro- tion of .50 for 
portion of .50 r = 0.00 (I) 
forr = 0.00 
(E) 
A +D 
A+D A+D 100 (—) — .5000 
f r — — .5000 N 
N N —— 
-5000 
.00 0000 5000 -0000 0.00 
05 0025 5159 0159 3.18 
10 0100 5319 0319 6.38 
15 0225 5479 0479 9.59 
20 0400 5641 0641 12.82 
25 0625 -5804 0804 16.09 
30 0900 5970 0970 19.40 
35 1225 -6138 1138 22.76 
40 1600 -6310 1310 26.20 
45 2025 6486 1486 29.72 
50 2500 -6667 1667 33.33 
.55 8025 6854 1854 37.07 
-60 3600 «7048 -2048 40.97 
65 +4225 1252 «2252 45.05 
.70 -4900 - 7468 2468 49.36 
175 5625 -7699 -2699 53.99 
.80 -6400 «7952 2952 59.03 
85 7225 +8234 -3234 64.68 
.90 -8100 8564 +3464 71.29 
+95 -9025 -8989 «3989 79.78 
1.00 1.0000 1.0000 -5000 100.00 
«aS. ese 


qo 
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Procedure 


The H entries in Table 1 were obtained from Pearson’s (1931) 
“Volumes of a Bivariate Surface,” and the H and J values were 
quickly derived from the H indices. As mentioned previously, the 
normal bivariate model was assumed to be appropriate. 


Findings 


From the entries in Table 1, one can quickly ascertain the nature 
of the relationship of the indices H, E, and I to both r and 7?. For 
example, when r = .60, or r? = .86, it is noted that the propor- 
tion H of valid positives and valid negatives—the proportion of cor- 
rectly placed individuals—is .7048. Thus on the criterion, the 
standing of slightly more than 7 out of 10 individuals is being ac- 
curately predicted. The complementary proportion 1 — H of .2952 
constitutes the proportion of “misses”, or incorrect placements. The 
index Æ with its value of .2048 for an r of .60 simply reflects the 
proportion in the total population accurately placed in excess of the 
chance proportion of .5000 which would be expected when the cor- 
relation is zero. This proportion of .2048 may also be viewed as 
representing an improvement I of 40.97 per cent (with a selection 
ratio of 1.00) in correct or valid placements over what would be the 
number of correct placements when the correlation is zero. 

It is also interesting to note that by interpolation a coefficient of 
about .71, which is associated with an 7? of nearly .50, also corre- 
sponds to a coefficient H of about .75—a value which represents an 
absolute gain of about .25 in the proportion Æ of valid inclusions 
and valid exclusions—or a magnitude in the improvement index 
I of nearly 50 per cent in terms of the number of accurate place- 
ments of individuals over the number when the correlation is zero. 
It can be seen that in the instance of a value of .71 in r, and in this 
instance alone, r? is approximately .50, which is equal to .01 J. In 
other words, for an r of about .71, the coefficient of determination 
converted to a percentage corresponds to the percentage of improve- 
ment J (approximately 50) in valid placements which takes place 
between an r of 0.00 and an r of 1.00. 

Since a coefficient of .71 is close to the upper limit of predictive 
validities in most personnel selection programs, it is apparent that 
for the median splits involved the proportion of correct placements 
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is not likely to exceed .75. Thus without a testing device, one 
could choose 2 out of 4 individuals correctly, but under optimal 
conditions in practical selection with tests, the greatest accuracy 
to be anticipated is that one could select 3 out of 4 individuals cor- 
rectly when the selection ratio is 1.00 and when the proportion of 
individuals successful on the criterion is .50. (Of course, with 
use of small selection ratios such as .10 or .20, proportions of correct 
placements could often approximate .80 or .85 for highly valid tests 
and for relatively difficult jobs as defined by rather small per- 
centages of 15 to 25 of individuals who have been judged to be 
successful on the criterion measure.) 

In Figure 2 the relationship between H and r as well as be- 
tween H and 7? is portrayed. Thus it is apparent that for the cus- 
tomary range of predictive validity coefficients between 30 and 
.65 the proportion of correct placements H varies between about 
.60 and .73. The complementary proportion of incorrect place- 
ments, or “misses,” known as false negatives and false positives, 
is given by 1 — .60 and 1 — .78 respectively, or by .40 and .27. The 
value of H is represented by the ordinate from the base line, 
whereas 1-H is given by the vertical distance from the horizontal 
boundary line at the top of Figure 2. 


Summary 


Largely for pedagogical purposes three indices have been proposed 
to explain the meaning of predictive validity. Use was made of the 
normal bivariate surface which was dichotomized at the median in 
both the predictor and criterion variables. The basic approach was 
that of relating three indices derived from the combined proportions 
of individuals in two cells along a diagonal of the fourfold table to 
values of the predictive validity coefficient r and of its coefficient of 
determination r?. 
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Tum Miller Analogies Test (MAT) continues to be one of the 
most widely used instruments for graduate school selection. A 
review of MAT research reveals a wide range of validity co- 
efficients. Values from .00 (Travers, 1948) to .69 (Gustad, 1950) are 
noted. Such discrepancies can in part be attributed to the consider- 
able variation in candidate qualifications and institutional differ- 
ences. The lack of reliable success criteria is also a significant con- 
tributor. The usual criterion of grades is generally conceded to lack 
sufficient consistency to allow for stable prediction, Very few valid- 
ity studies can be located which involve the use of objective criteria, 
such as standardized instruments or end-of-program comprehen- 
sive examinations. The purpose of this paper is to report data 
which compare the predictability of the MAT relative to the criteria 
of grades and comprehensive examination scores. 


Variables 


The course work leading to a Masters Degree in Education at 
a large urban university requires, in addition to the usual electives 
and prescribed major instruction, a minimum of one course in 
each of the following areas, Educational Psychology (EP), Meas- 


1Data analysis was completed with support of National Science Foundation 
Grant GP-1137 made to Syracuse University Computing Center. The assist- 
ance of Miss Agnes R. Esposito and Mr. James G. Black in the completion of 
this study is gratefully acknowledged. 
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urement and Statistics (MS), and Cultural Foundations of Educa- 
tion (CF). Students frequently take more than one course in each 
of the core areas. The following subjective estimates of success 
(using a threepoint scale) were derived from each student’s tran- 
script: 


1. Mean grade in Educational Psychology core courses (EPG) 

2. Mean grade in Measurement and Statistics core courses (MSG) 
3. Mean grade in Cultural Foundations (CPG) 

4, Total Master’s grade point average (TG) 


Chansky (1964) recently noted that since grades do not meet the 
assumptions underlying a ratio or interval scale of measurement, 
the “mean GPA” tends to be relatively meaningless. He suggested 
that a median GPA be used whenever grades were used as cri- 
teria. The final grade criterion was, therefore: 


5. Median grade in total Master’s program (MdTG) 


Master’s candidates are required to pass a 285-item comprehen- 
sive examination which equally represent the three core areas, Ap- 
proximately 80 per cent of the items are multiple-choice; the re- 
mainder, true-false and matching. The examination was judged 
to have high content validity, as the items were written by the 
core area instructors and then evaluated by test construction ex- 
perts. Part and total scores formed the basis for four more criteria 
(with Kuder Richardson Formula 21 reliabilities—subsequently ab- 
breviated as KRe:;—in parentheses) : 


6. Educational Psychology subtest score (EPS) (.47) 
7. Measurement and Statistics subtest score (MSS) (.82) 
8. Cultural Foundations subtest score (CFS) (.65) 


9. Total Master’s Comprehensive Examination 
Score (TMCS) (.84) 


Form K of the Miller Analogies Test (MAT) (KRa = .91) was 
administered when students applied for graduate work. The distri- 
bution of times between MAT and completion of degree was posi- 


tively skewed, with a mean of 23 months and a median of 20 
months, 
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Sample 


| The sample was composed of 115 males and 104 females. Since no 
sex differences were noted, the groups were combined (N = 219). 
Mean age of the total group was 30 years. All students not only 
had taken the MAT but also had completed the Master’s degree be- 
tween July 1958 and March 1963. 


| Results and Discussion 


Means, standard deviations, and correlations of variables 
involved in the study are presented in Table 1. The greater effec- 
TABLE 1 
Correlations, Means, and Standard Deviations for MAT 
1 and Criterion Measures (N = 219) 
Masters Comprehensive 
Grade Criteria Exam Scores 
Variables [MSG CFG TG MaTG| [eps Mss CFS TMCs| MAT & s 
1. EPG 837 25 .48 46 40 .45 89 2.17 58 
2. MSG 23 .55 .55 4l 64 36 4 48 2.15 64 
3. CFG 40 84 23 20 .24 27 05 1.91 59 
4. TG 98 44 .56 38 54 26 2.32 29 
5. MdTG „44 57 34 55 .28 2.33 33 
6. EPS 54 48 75 A2 50.41 6.61 
7. MSS 51 87 46 59.12 10.82 
8. CFS 77 44 46.21 8.17 
| 9. TMCS 1 155.61 21.13 
10. MAT 47.32 16.70 


a Symbols for the variables are explained in the text. 
b Mean of the median grade point averages. 
tiveness of the MAT in predicting Comprehensive Examination 
scores than grades is noted. This finding, in part, could be expected 
when one considers the skills required for successful performance 
in the classroom versus those involved in objective written examin- 
ations. The difference in validity coefficients reflects, perhaps, the 
| lack of uniform grading practices. Such a lack is particularly true 
| for the Cultural Foundation (CF) grades. The correlations of 
CF grades with all variables are moderate to low. Such a pattern of 
correlations might indicate the presence of a departmental bias in 
grades which lacks substantial relationship to current academic 
criteria being used in other departments. An examination of the 
means and standard deviations of the grade criteria might also be 
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interpreted as evidencing a ceiling effect, which is associated with 
lower predictability of the MAT. The correlation of .54 between 
total grade average and the total scores on the Comprehensive 
Examination, would indicate that different instructional outcomes 
are being evaluated. Further evidence bearing on differences in 
grading practices is found in the moderately low intergrade correla- 
tions. 

The correlation of .98 between mean and median grade point 
average would indicate that Chansky’s caution is of no practical 
consequence. 

The correlations of .26 between MAT and grades and of .51 be- 
tween MAT and Comprehensive Examination scores, are typical of 
those found in the literature. They are interpreted as justifying the 
continued use of the MAT, although additional predictive valid- 
ity studies should be periodically undertaken. 
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Problem 


Tus study was undertaken to determine the predictive validity 
of the National League For Nursing, Pre-Nursing and Guidance 
Examinations for success in a three year diploma program in a 
small private hospital school of nursing. 


Variables 


The National League For Nursing, Pre-Nursing and Guidance 
Examination (PNLN), a battery made up of different tests of the 
Cooperative Test Division of Educational Testing Service, is a 
widely used battery for selection of students in diploma programs. 
Taylor and others (1963) in their extensive review of selection 
and recruitment practices in nursing education reported that 288 
of 698, or forty one percent, of diploma schools sampled use the 
PNLN. Forty seven percent of these ranked it as the most im- 
portant procedure used in selection. 

The PNLN covers five areas thought to measure some of “the 
capabilities and proficiencies which are related to the development 
of registered nurse competencies and which might reasonably be ex- 
pected of high school graduates” (National League For Nursing, 
1961, p. 3). These are: academic aptitude, reading comprehension, 

1This work was done in part at the MIT Computation Center, Cambridge 
Massachusetts. 
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mathematics, natural sciences, and social studies achievement. 
These five tests yield the following nine scores: (1) Verbal Apti- 
tude, (2) Quantitative Aptitude, (3) Total Aptitude, (4) Speed of 
Comprehension, (6) Mathematics Achievement, (7) Natural Science 
Achievement, (8) Social Studies Achievement and (9) a Composite 
Score which is a weighted combination of the scores on the indivi- 
dual tests. 

Taylor (1963, p. 2) uncovered only three published multiple cor- 
relation studies of the predictive validity of the PNLN. With State 
Boards as the criterion the R’s ranged from .61 to .70. When 
grades were used as the criterion, the obtained R was .57. 

Criteria 

The criteria for success in the three year diploma program were 
threefold. First, grades in individual courses in the freshman 
year were used. Unlike reported studies (Taylor, 1963), it was de- 
cided neither to lump all courses together nor to utilize an over-all 
grade point average. This decision was made so that the predictive 
capacity of the PNLN for clinical and theory courses could be 
compared, and further to determine the forecasting power of the 
battery for the different types of science courses, ranging from 
Pharmacology to Psychology, which the student nurse must master. 

Second, the scores of students on four of the Psychological Cor- 
poration’s Achievement Tests in Nursing were used as criterion 
measures. These achievement tests, in the areas of Anatomy and 
Physiology, Chemistry, Microbiology and Pharmacology were 
taken by students upon completion of course work in each of the 
areas measured by those tests. 

The third and final criterion of success was performance after 
graduation on State Board Examinations. These tests in the areas 
of Medical, Surgical, Psychiatric, Pediatric and Maternal nursing 
comprise the licensing examination for nurses in the Common- 
wealth of Massachussetts. 


Statistical Methods 


The data were subjected to an inductive multiple regression analy- 
sis.2 The program which was used instructed the computer to se- 


2 Acknowledgment is made to Dr. Albert Beaton, Jr., of Educational Testing 


Service for the development of the Computer Program for the IBM 7094, 
which was used in this study. 
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lect from the independent variables those which contribute to the 
multiple correlation in the order of their importance. Under this 
system, the computer first chooses that variable which correlates 
most highly with the criterion and then proceeds in order to select 
the next highest correlation with the effects of the first variable par- 
tialed out, and then the next highest multiple correlation with 
the first and second variables partialed out and so on. 

The author tested to see whether the additional variables in the 
multiple regression equation led to a significant increase in the ac- 
curacy of prediction over the correlation obtained with increasingly 
smaller subsets of the same variables, using the formula for such 
a test found in McNemar (1960, p. 279). 

Whenever the multiple correlation technique was utilized on 
the PNLN the total aptitude and the composite scores, being com- 
binations of other tests in the battery, were not included in the 
analysis. Zero order correlations were computed between those two 
combination scores and the various criterion measures. 


Subjects 


The subjects were student nurses admitted to a small three year 
diploma program run by a private hospital in Massachusetts be- 
tween the years 1961 and 1963. 

First year grade point averages were available for 90 students. 
State Board results and Psychological Corporation’s Achieve- 
ment Test in Nursing scores were available for 36 students. 


Results 


Table 1 presents the zero order correlation coefficients between 
the PNLN and the criterion measures of grade point average in 
various first year nursing courses. 

Fifteen of the correlations in Table 1 are significant at the .01 
level of confidence, while the remaining sixty six correlations 
did not differ significantly from a correlation of 0.00. 

The Verbal and Quantitative tests add nothing of significance to 
prediction when used in a multiple regression analysis. The Math- 
ematics and Social Studies tests also add nothing to prediction 
when the battery is analyzed multivariately. Only one obtained 
multiple correlation (R = .30), between the criterion grade 
point average in Anatomy and Physiology and the PNLN vari- 
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ables of Speed of Comprehension, Level of Comprehension, and 
Natural Science, was superior in forecasting power to the highest 
zero order correlation obtained. 

When the Psychological Corporation’s Achievement Tests in 
Nursing were used as criterion measures, only one correlation, mul- 
tiple or zero order was obtained which differed significantly from a 
correlation of 0.00. This was an r of .37 between the PNLN Mathe- 
matics Achievement test and the Microbiology Achievement test 
of the Achievement Tests in Nursing of Psychological Corporation. 

Table 2 shows the zero order correlation coefficients between 
the PNLN tests and the criteria of performance on State Board 
Examinations taken three years after administration of the PNLN. 
The correlations in the last column, pass-fail, are point biserial 
correlation coefficients; the remaining are zero order correlations. 


TABLE 2 


Correlation Coefficients between the National League For Nursing 
Pre-Nursing and Guidance Examinations and 
Massachusetts State Board Examinations 


Pre-Nursing Massachusetts State Board Examinations 
and Guidano ——@_—_—_—_ 
Examinations Psychi- Pass- 

(PNLN) Medical Surgical atric Pediatric Maternal Fail 


Speed of Com- 


prehension 42 .60 ae ae = AL 
Level of Com- 

prehension ae 55 oe bd ee 45 
Mathematics nid sae .20 52 34 39 
Natural Science a 34 Sng she ae =F 
Composite 45 62 42 .33 ae 43 


* Obtained Correlations are not Significantly Different from a correlation of 0.00 


Fifteen of the 30 correlations in Table 2 are statistically signifi- 
cant at the .05 level of confidence or greater; the remaining fifteen 
exhibited only chance relationships. Using the seven tests multi- 
variately added nothing in the way of statistically significant in- 
crease in prediction over the results obtained in Table 2. The 
Verbal, Quantitative, Total, and Social Studies tests had no rela- 
tionship to any of the criteria, and therefore do not appear in Table 
2. The Composite score did correlate with all but the Maternal test 
of the State Boards. 
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Discussion 

The Verbal, Quantitative and Total scores which constitute 
the academic aptitude section of the PNLN have no statistically 
significant predictive validity for any of the criterion measures uti- 
lized with this sample, Although this is understandable in the 
clinical areas, in which performance is non-scholastic in nature, the 
complete lack of relationship with nursing theory and college level 
science courses is not readily explainable. 

None of the tests of the PNLN has predictive validity for 
clinical performance as measured by grades in courses such as Fun- 
damentals of Nursing Practice, Medical-Surgical Practice, and 
Pharmacology II. Tests must be found or developed to measure 
aptitude in these practical, clinical, performance-oriented courses. 
Behavioral analysis of the clinical duties of registered nurses would 
be an excellent starting point in the development of such instru- 
ments. 

The Natural Science and Reading tests appear to be the best 
predictors of theory courses in nursing and science courses. Al- 
though statistically significant, the obtained correlations are of 
dubious practical value. Multivariate analysis adds little or nothing 
to prediction. This undoubtedly is due to the relatively-high inter- 
correlations between the tests in the PNLN battery. A tailor 
made battery for nursing theory and clinical practice aptitudes 
should contain tests with low intercorrelations and high indivi- 
dual correlations with the criteria. This is not the case with the 
PNLN for this sample. 

The PNLN did have moderate long range predictive validity 
when used to forecast performance on the State Board Examina- 
tions. The sample was small, thirty-six, and homogeneous because 
of attrition over the three year span. In spite of this restricted range 
which tends to suppress correlations, the PNLN Composite score 
was related to all State Board Tests with the exception of the 
Maternity Nursing Examination. The Academic Aptitude Tests 
and the Social Studies Tests had no predictive validity with this 
criterion, whereas the remaining achievement tests exhibited mod- 
erate long-range predictive powers. 


> 
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Conclusion 


Although certain subtests of the PNLN did correlate signifi- 
cantly with various nursing theory and science courses, the relation- 
ships were not of an order sufficient for decision making regarding 
admissions. The crucial area of clinical nursing course perform- 
ance was unrelated to antecedent performance of any of the tests 
in the PNLN battery. 

Certain Subtests of the PNLN did exhibit long-range predic- 
tive validity for this sample for later performance on State Board 
Examinations. 
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MULTIPLE DISCRIMINANT PREDICTION OF MAJOR 
FIELD OF STUDY 


ROBERT F. STAHMANN ann NORMAN E. WALLEN 
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“Mouurrerm discriminant analysis is a statistical method of com- 
bining test scores or other data so as to maximize the differences 
between the groups and minimize the differences within each group 
(Dunn, 1959, p. 15).” The discriminant analysis, first used as a 
method of classifying individuals into one of two groups, was in- 
troduced into psychological literature in the late 1930’s (Travers, 
1939). This technique of classification into one of two groups, has 
been applied to various academic classification problems (Selover, 
1942; Baggaley, 1947; Tiedeman and Sternberg, 1952). The dis- 
criminant analysis technique has evolved to become a technique 
which may be used in classifying individuals into one of several 
groups. 

The multiple discriminant analysis has been applied to the prob- 
lems of predicting academic group membership (Christensen, 1953) 
and guidance of students based upon such predictions (Stinson, 
1958). Dunn (1959) found the discriminant analysis to be supe- 
rior to the regression analysis for prediction of academic major field. 
Tatsuoka (1957) applied the multiple discriminant, multiple re- 
gression, and “joint-probability model,” which was a hybrid of the 
discriminant and regression analyses, to a sample of students and 
concluded that for the groups under study the discriminant analy- 
sis predicted correctly the greatest number of actual fields of study. 

Based upon a multiple discriminant analysis of individual en- 
trance examination data, the present study was an attempt to 
classify or predict major field of study at graduation for a sample 
of university students. 
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Method 


University of Utah graduates of 1962, 1963, and 1964, in selected 
major fields of study served as the population from which the 
samples were drawn for inclusion in the study. The study re- 
quired two samples, an experimental sample upon which the 
discriminant-analysis linear functions were computed, and a cross- 
validation sample upon which the predictions as to major field 
of study were made. For the cross-validation sample use was 
made of the functions found from the discriminant analysis of 
the experimental sample. A sample size of 50 was desired for each 
experimental and each cross-validation sample. Random assignment 
was made to the experimental and cross-validation samples for 
the fields in which the original number of available subjects ex- 
ceeded 50. In each of those fields in which the original number of 
available subjects was less than 50 all subjects for that field were 
included in the experimental sample. The prediction as to major 
field of study was run separately for men and women students. 

The analysis was run for each problem through using the experi- 
mental sample as a normative sample upon which the discrimi- 
nant weights were computed, and the cross-validation sample upon 
which the predictions as to major field of study were made. A 
secondary analysis of the experimental samples was made through 
classifying each individual in that sample on the basis of the dis- 
criminant analysis of the same sample. 

The entrance examination battery which made up the data used 
in the study consisted of the following measures: (1) Coopera- 
tive English Test: Lower Level, Forms RX, T and Z; (2) Co- 
operative Mathematics Pre-Test for College Students, Forms X and 
Y; (3) Cooperative General Achievement Tests: Test 2, Natural 
Science, Form YZ; (4) Occupational Interest Inventory, 1956 Re- 
vision, Advanced level. 


Results 


The discriminant analysis and classification were facilitated by 
the use of a standard computer program. The data from the male 
experimental sample were submitted to the multiple discriminant 
analysis. The linear discriminant functions for the males were 
then used in the classification of individuals in the male cross- 
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validation sample. The discriminant functions obtained from the 
analysis of the male experimental sample were read into the com- 
puter and used as the discriminant weights for predicting major 
field for the male cross-validation sample. Hach individual was 
predicted as belonging to the field for which the discrimination 
score was the highest. The results of the prediction for the male 
cross-validation sample are summarized in Table 1. 


TABLE 1 


Agreement between Predicted and Actual Field of 
Study, Male Cross-Validation Sample 


Predicted Major Field 
Actual Major Letters & 
Field Engineering Business Pharmacy Science Total 

Engineering 29 13 5 50 
(17) (10) (8) (15) 

Business 8 3 14 14 39 
(18) (8) (6) (12) 

Letters & Science 11 11 5 23 50 
(17) (10) (8) (15) 

TOTAL 48 27 22 42 139 


Note—Top numbers refer to predictions made by the discriminant analysis. Numbers in paren- 
theses are the number of classifications expected by chance, based on marginal totals. 


The numbers of correct classifications expected by chance, based 
on marginal totals, are shown as the numbers in parentheses in 
Table 1. The frequencies arising from predictions for the male 
cross-validation sample based upon the weights of the male experi- 
mental sample exceed the numbers expected to be correctly pre- 
dicted by chance for the engineering and the letters and science 
fields of study. The frequency in the classification for the business 
field is below the number expected by chance. 

The data from the female experimental sample were submitted 
to the multiple discriminant analysis, and the discriminant 
weights which were obtained were applied to the predictions for 
the female cross-validation sample. The prediction as to major field 
of study for the female cross-validation sample placed 32 of the 50 
female elementary education majors in the correct classification. 
This prediction is significantly better than the chance prediction 
of 16. The results of the predictions for the female cross-validation 
sample based upon weights of the female experimental sample are 
summarized in Table 2. 
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TABLE 2 


Agreement between Predicted and Actual Field of 
Study, Female Cross-V alidation Sample 


Predicted Major Field 
Actual Major Elementary Letters & 
Field Nursing Education Science Total 
Elementary 8 32 10 50 
Education (16) (16) (16) 


Note—Top numbers refer to predictions made by the discriminant analysis. Numbers in paren- 
theses are the numbers of classifications expected by chance, based on marginal totals, 


The computer program which was available for this study was 
written for use primarily as a classification program setting up the 
discriminant weights on, and predicting for, the same sample. Such 
an analysis was run and included in this study. Table 3 summarizes 
the results of the discriminant analysis of, and prediction for, the 
male experimental sample. An examination of the results of this 
prediction reveals that the frequencies associated with predictions 
for each of the four major fields of study exceed chance expectations. 


TABLE 3 


Agreement between Predicted and Actual Field of Study, Male Experimental 
Sample Using Functions of Male Experimental Sample 


Predicted Major Field 
Actual Major Letters & 
Field Engineering Business Pharmacy Science Total 
Engineering 38 9 2 1 50 
f (15) (13) (8) (14) 
Business 3 25 8 3 39 
(12) (10) (6) (11) 
Pharmacy 5 3 13 8 29 
(9) (7) (4) (8) 
Letters & 5 6 4 35 50 
Science (15) (13) (8) (14) 
TOTAL 51 43 27 47 168 


Note—Top numbers refer to predictions made by the discriminant analysis. Numbers in paren- 
theses are the numbers of classifications expected by chance, based on marginal totals; 


A similar analysis, based upon functions of the female experi- 
mental sample, and predicting for the members of the same female 
sample was run. Table 4 summarizes the results of the analysis of, 
and the prediction for, the female experimental sample. The fre- 
quencies associated with predictions for each of the fields of study 
exceed chance expectations. 
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TABLE 4 


Agreement between Predicted and Actual Field of Study, Female Experimental 
Sample Using Functions of Female Experimental Sample 


Predicted Major Field 
Actual Major Elementary Letters & 
Field Nursing Education Science Total 
Nursing 3 5 5 43 
(16) (16) (11) 
Elementary 9 34 6 49 
Education (18) (18) (13) 
Letters & 7 10 24 41 
Science (15) (15) (11) 
TOTAL 49 49 35 133 


Note—Top numbers refer to predictions made by the discriminant analysis. Numbers in paren- 
theses are the number of classifications expected by chance, based on marginal totals. 


The analysis-of-variance technique was applied to the male 
experimental and cross-validation samples and to the female ex- 
perimental sample to examine the differences among the means of 
the 20 variables for each of the major fields of study. These analy- 
ses showed the greatest variability among the means of the interest 
test rather than among the means of the academic achievement 
measures. 

The ¢ test was applied as a test of equivalence of samples. This 
analysis revealed that there were no significant differences (.05 
level) between the experimental and cross-validation samples on 
18 of the 20 variables in any major field. 


Summary 


Multiple discriminant analysis as a method of predicting in 
which one of several groups an individual is most likely to place 
was applied to the problem of predicting major field of study at the 
University of Utah. Variables used in the study consisted of inter- 
est and achievement measures and of an index of urban or rural 
high school attendance, all of which were obtained from the fresh- 
man entrance examination battery of the students. 

Generally, the question as to whether the freshman entrance bat- 
tery at the University of Utah was shown to be an effective predic- 
tor of major field of study at graduation can be answered in the af- 
firmative. The prediction for the business field of study was below 
chance expectation—an outcome tending to support the finding 
of Christensen (1953) in which prediction at chance was made for 
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a pre-business group. These findings indicate that in attempting to 
predict membership in a heterogenous group such as business stu- 
dents, one must perhaps use variables other than achievement and 
interest measures to achieve adequate discrimination. 
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AN ATTEMPT TO VALIDATE AN EMPIRICALLY-DERIVED 
INTEREST SCALE AND STANDARD KUDER SCALES FOR 
PREDICTING SUCCESS IN HIGH SCHOOL GEOMETRY 


GERALD S. HANNA 
University of Alaska 


Problem 


Tus paper describes an unsuccessful attempt to measure some of 
the unaccounted for variance in criteria of learning which, as 
Jackson and Strattner (1964, p. 518) recently observed, lingers 
after the effect of ability, prior learning, teaching method, and 
other task-related variables have been removed. This variance has 
interested researchers for some time and continues to stimulate re- 
search. 


Earlier Studies 


Super and Crites (1962, pp. 479-81) cited several studies in 
which significant relationships existed between scales of the Kuder 
Preference Record (Vocational) and academic success in various 
fields. Although none of these studies pertained directly to geome- 
try, it appeared likely that similar findings would occur if geometry 
were investigated. They concluded that the Kuder seems “to have 
real possibilities even for the prediction of success in courses, for 
scores are significantly related . . . to grades in some appropriate 
subjects, specifically the scientific, mathematical, and literary” 
(Super and Crites, 1962, p. 491). 

The only study located which dealt directly with geometry pre- 
diction was Townsend’s (1945) in which a correlation of .31 was 
obtained between the Strong Vocational Interest Blank for Men 
(Mathematician Scale) and the Cooperative Plane Geometry Test 
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among 63 private secondary school boys. The Mathematics-Science 
Teacher Scale correlated .28 with the criterion in the same sample. 


The Present Study 
Standard Kuder Scales 


In a recent investigation (Hanna, 1965), numerous predictive 
variables were obtained for 226 geometry students at the beginning 
of a school year. Variables included many cognitive measures, past 
grades in related courses, standard interest scales as well as an 
empirically-derived interest inventory scale, and pupils’ predic- 
tions of how well they would achieve in the subject. Criteria in- 
cluded an achievement test for the 202 pupils remaining at the end 
of the semester, teachers’ rankings of students on the semester’s 
work, and a measure of improvement in deductive reasoning in non- 
geometric situations. Multiple regression equations were derived us- 
ing selected combinations of predictors and a composite criterion. 

Correlations of the ten standard scales of the Kuder, Form CM, 
with the composite criterion ranged from —.17 to .05; the median 
absolute value was .08. Only one scale (scientific) correlated sig- 
nificantly (but negatively!) at the .05 level with this criterion. 
Improvement of multiple regressional prediction by use of ex- 
isting Kuder scales was equally disappointing. The hope expressed 
by Super and Crites that existing Kuder keys would prove useful in 
predicting course success—a use for which the scales were not 
derived—was not realized in this investigation. 


Empirical Scales 


A stratified random subsample of 94 Kuder answer sheets was 
sorted so that top and bottom 27 per cent groups could be separated 
on the basis of the composite geometry criterion. These groups were 
then contrasted on every response position for each Kuder item. 
The ratio used for item discrimination was the difference between 
the number of responses in the top and bottom groups for a given 
response position, divided by the average number of responses in 
that response position for the two groups. Response positions having 
a discrimination ratio of .4 or more were employed if, and only if, 
the difference in the actual number of subjects who marked the re- 
sponse position was two or more. For items in which the differ- 
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ence in the number of pupils who marked the response position was 
four or more, the item was used in the empirical key, regardless of 
its discrimination index. Separate keys were made for the positive 
and negative discriminators. The score for each pupil was deter- 
mined by subtracting the score obtained when the key of negative 
discriminating items was used, from the score obtained when the 
key of positive discriminators was used. The derived keys were then 
cross-validated with the other 108 subjects by correlating the 
composite criterion scores with the scores obtained from the em- 
pirical keys. 

The empirical key correlated .50 with the composite criterion 
before allowance for shrinkage was made. In cross-validation of 
this key with the answer sheets of the pupils with which it was not 
derived, the correlation was only .27. This validity value was used 
in derivations of multiple regression equations. 

Predictors were added by a stepwise regression computer pro- 
gram. In addition to the empirically derived scale, all ten standard 
Kuder scales were available for computer selection. Successive 
additions are reported until the next variable would add consider- 
ably less than 1 per cent to R?. Equation (1) reports the multi- 
ple regression equation used for a very productive combination 
of readily obtainable raw data wherein R = .59. Equation (2) gives 
the weights of the same data when used in combination with the 
standard and empirical Kuder scales; this equation correlated 
.63 with the composite criterion. 


(1) R = 53V + .53W + .50X — .35Y + .09Z 
(2) R= .52V + 57W + 47X — 40Y + .04Z — .02L + .03M 
+ 09N 
where, 
= cumulative marks in first year algebra (5 point scale) 
W = cumulative marks in last year of general mathematics 
studied (5 point scale) 
= subjects’ own prediction of their geometry marks (5 point 
‘scale) 
year of high school (4 point scale) 
Differential Aptitude Test: Verbal Reasoning plus Nu- 
merical Ability raw scores 
= Kuder Mechanical Scale raw score 
= Kuder Persuasive Scale raw score 
= Kuder Social Service Scale raw score 
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Conclusions 


Neither standard nor empirically derived Kuder scales contributed 
significantly to multiple regression prediction of geometry success, 
as is evident by their slight weight or absence from equation (2). 
Apparently, the independent variance which interest factors contrib- 
uted to multiple regression equations had been tapped about as 
well as present instrumentation would allow by the more easily- 
obtained algebra marks, arithmetic marks, subjects’ predictions 
of geometry marks, and year in high school. 

It appears from this study that efforts to develop empirical keys 
to the Kuder Preference Record for geometry prediction would 
not offer promise. 

° 
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VALIDATION OF THREE TESTS OF COGNITIVE STYLE IN 
VERBALIZATION FOR THE THIRD AND SIXTH GRADES 
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ALTHOUGH there exists a history of research in qualitative styles 
of thinking and verbalization—abstract, functional, and concrete— 
there are still many research needs in this area. Investigations of 
qualitative levels of thinking may be traced to Piaget’s theoretical 
work on developmental sequences in concept formation. Russell 
and Saadeh (1962) summarized the research through 1961 and 
Lundsteen (1966b) through 1964. From these two articles the fol- 
lowing needs for research were implied: (1) the desirability of ascer- 
taining the nature and degree of relationship or congruent validity 
between experimental measures of qualitative levels of verbalization 
and other measures purporting to represent verbal abilities—typi- 
cally standardized tests used in schools; and (2) the development of 
stimulus material in the form of paragraphs or stories that would 
tap higher level processes or operations involved in concept forma- 
tion as well as critical and creative thinking. These operations, 
in Guilford’s (1961) terminology, involve products in a hierarchial 
order of complexity. 


Problem 


Accordingly, the major task of this research was to investigate 
the degree and pattern of interrelationships among three experi- 
mental measures intended to reflect three qualitative levels of verbal 
thinking, a test of critical listening, a standardized test of reading, 
and a standardized test of verbal ability. 

Furthermore, it was hoped that study of performance on the 
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experimental measures of verbal function at differing age levels 
would contribute further evidence regarding their validity in rela- 
tion to other measures of language behavior. 


Hypotheses 


With the above purposes in mind four major hypotheses were 
formulated: 

1. First, it was hypothesized that if the construct of qualitative 
levels of thinking—abstract, functional, and concrete—was valid, 
each of the levels would be correlated highly and positively across 
all constructed (experimental) measures of the same planned quali- 
tative level. In other words, a high relationship would exist for 
each of the separate scores for responses designed to be qualita- 
tively different regardless of the stimulus content. 

2. Second, there would be a higher relationship between pairs 
of tests, in which qualitative levels in verbal ability were care- 
fully delineated, than between each of these same tests and other 
measures of verbal ability without built-in qualitative levels. 
It was hypothesized that although the relationship of qualitative 
to non-qualitative measures (as represented by a listening test 
and two well known standardized tests) would be positive and sub- 
stantial, it would not be so high that one type of measure could 
be validly substituted for the other. 

3. Third, it was hypothesized that mean differences between 
scores in tests of words, of paragraphs, and of stories reflecting de- 
yelopmental levels in qualitative thinking at grades three and six 
would indicate the same preferences (modal preferences) for cer- 
tain qualitative levels as had been found in earlier studies (Russell 
and Saadeh, 1962) in which only word stimuli had been used. 
Specifically, while the third-grade pupils in comparison with the 
sixth-grade pupils would tend to show significantly higher means 
for the concrete score, these third-grade pupils would tend to show 
significantly lower means on the abstract score. Moreover, there 
would be no significant difference on the average functional score 
between the two grade levels. 

4. Fourth, according to the progressive complexity and length 
of the stimulus material (word, paragraph, or story) in each of 
the qualitative measures, there would be differences at each of the 
two grade levels in the relative degree of preferences for abstract, 
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functional, and concrete levels of verbalization—i.e., in the cogni- 
tive styles demonstrated. It was hypothesized that the more com- 
plex materials in the paragraph test and in the story test would 
be accompanied by an increase in the employment (that is, a re- 
gression to) of a concrete cognitive style. (The arbitrary criterion 
used to define a cognitive style of a pupil was as follows: at least 
40 per cent of the responses in the one dominant qualitative category 
and for each other category at least a per cent of 10 less than that 
found in the dominant category.) 


Definitions 

The categories—abstract, functional, and concrete—have been 
defined elsewhere (Reichard, Schneider and Rapaport, 1944; Russell 
and Saadeh, 1962; Lundsteen, 1966b). Nevertheless, a brief de- 
scription of each of these qualities is presented: 

Abstract represents a style or quality of verbal expression in 
which the pupil discriminates by the special features of a class. 
The child’s concept has grown out of common features toward 
accuracy and breadth of definition. This verbalization is not an 
individual instance, not isolated, not particular or personal. 

Functional characterizes verbal expression using discrimination 
of objects or situations by use to the child, or by what the child 
does. The attitude implicit in the verbalization may be even some- 
what opportunistic, self-centered. This style is less encompassing, 
but not quite so particular as the next level. In the earlier version 
of the word stimulus test (Russell and Saadeh, 1962), the func- 
tional category actually included abstract classification involving 
discrimination by assignment to the operational class of the object, 
the use or function of the object, not operations of the child him- 
self. This aspect of the wording of responses appeared to effect an 
undesirable kinship between the abstract and functional categories. 
Accordingly, the functional responses were revised and constructed 
in keeping with the pupil-centered definition as stated earlier by 
Reichard, Schneider, and Rapaport (1944). 

Concrete represents a style involving discrimination by non- 
essential details, incidental features, and isolated particulars often 
of a highly sensory nature. Individual instances are given as op- 
posed to the distinguishing features of the class. Particulars may 
involve size, color, time, movement, weight, number, location, and 
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origin. If the definition involves a “social” concept, i.e., “anger, 
the child might refer to tone of voice or bodily position. The defini- 
tion may use the personal words “I” or “my.” (The error category 
represented a mistaken idea, completely irrelevant, or misinforma- 
tion.) 


” 


Method 
Statistical Treatment 


This investigation, classed as a descriptive study of validity, 
attempted to examine nominal categories and interrelationships be- 
tween test variables. Through use of correlation programs at the 
Computing Centers, University of California, Berkeley and Santa 
Barbara, product-moment correlation coefficients were computed 
among all possible pairings of variables cited in Table 1. Investiga- 
tion of development of abilities in qualitative thinking was effected 
by applying t tests to differences between grade-level means for each 
of the qualitative scores for each of the three experimental measures. 
(See Table 2.) 


Sample 


The subjects for this investigation were 178 children, com- 
pleting all six tests (to be described) in four third-grade and four 
sixth-grade classes in Goleta, California. From the parent popula- 
tion, consisting of six schools with 22 classes (678 pupils), the 
eight classes for the investigation were chosen randomly. The 
Goleta Union School district, classified as having a typical socio- 
economic distribution for suburban schools (Jones, 1965), included 
10 elementary schools with kindergarten through sixth grade. The 
sixth-grade sample appeared to be somewhat representative of the 
general population of sixth grades because of its close approximation 
to norms on the reading portion of the Sequential Tests of Educa- 


tional Progress (STEP Reading) and the School and College Ability 
Tests (SCAT). 


Experimental Measures 


To give a general description, a test battery was constructed of 
variables which would presumably measure levels of thinking 
(abstract, functional, and concrete) when the responses sought 
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were stimulated by (1) the word, (2) the paragraph, and (3) the 
unfinished problem-story. An additional hope was to measure proc- 
esses of thinking as theorized by Russell (1956). P 

The instrumentation may be described specifically, measure by 
measure, as follows: 

Concept Formation, Word Test. First, to measure qualitatively 
the process of concept formation, or definition with a one word 
stimulus, a multiple-choice test of 42 items with constructed re- 
sponses for abstract, functional, and concrete qualities was used. 
This measure, described elsewhere in detail (Russell and Saadeh, 
1962; Lundsteen, 1966b), and revised especially for this study, 
showed a test-retest reliability of .78. 

Creative and Critical Paragraph Test. Second, the newly de- 
veloped test of creative and critical paragraph reading contained 
in five paragraphs elements of mood, humor, fantasy, and facts. 
It was hypothesized that the nature of the content made it possible 
for mental operations of the pupils to approximate Russell’s theo- 
rized processes of evaluative and creative reading. Or, in Guil- 
ford’s (1961) terminology, operations involving relations, implica- 
tions, and transformation were hypothesized as being measured. 
The 42 words used as stimuli in the test mentioned above were 
embodied in the five paragraphs in the second test. Examples of 
questions were inferential types as to character relationships, and 
critical types, such as, evaluating the mode of presentation as fact 
or fancy. Forty-five questions, approximately evenly distributed 
between the paragraphs, dealt with this type of question. Again, as 
in the first test, an attempt was made to construct the multiple 
responses in abstract, functional, and concrete styles of language. 

Twenty graduate students from the University of California, act- 
ing as judges, aided in examining the validity for the response 
categorization. Test-retest reliability for this measure was .60 
(N = 88). Percentages of preferences for the three styles at each 
grade level for each item were also examined. 

Problem-Story Test. Third, the experimental test of the problem- 
solving process developed for this study consisted of 48 questions. 
There were four different unfinished stories or problems with 12 
types of questions accompanying each story. The 12 types of ques- 
tions, derived from the literature on problem solving, included items 
grouped under the categories of (1) inference of the problem, (2) 
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hypotheses for solving the problem, (3) procedures in solving the 
problem, and (4) evaluation of an hypothesis. 

The problem areas for the stories included the following: coping 
with change, integration into a new peer group, and the rift be- 
tween the old and the young. When the story reached a point 
where the problem could be inferred, the selection terminated and 
the first of twelve question types was asked. This version of the 
problem-solving test contained the multiple-choice answers rep- 
resenting, again, the abstract, functional, and concrete quality 
of verbalization. 

During two separate item revisions, two classes of 20 to 30 
graduate students at the University of California, Santa Barbara, 
were used in judging the phrasing of the responses on this test. The 
test-retest reliability was .78 (N = 88). Percentages for each qual- 
ity under each item for each grade level were also inspected. 

Each of the three experimental tests was scored by finding for 
each individual how many items he marked with responses keyed 
as abstract, functional, and concrete. For example, in the test of 
concept formation with the word as a stimulus, an individual 
might have answered 14 responses keyed as abstract, 12 as func- 
tional, 8 as concrete, and 8 as errors. 

Although all three tests need further revision, the results 
from the present administration seemed of sufficient interest to re- 
port. More measures need to be constructed to represent the 
process variables. 


Additional Measures 


Three additional tests were administered to the sixth-grade 
sample. The test of critical listening has been described elsewhere 
(Lundsteen, 1963, 1964, 1965, 1966a). This instrument, with an 
estimated test-retest reliability of .72, measures ability to analyze, 
interpret, and give reasons for judging a speaker’s purpose, propa- 
ganda, and argumentation. It was thought that a test of listening 
ability at higher levels of mental operation would be positively 
related to the newly constructed measures, Test administration 
was effected by use of a tape recorder and IBM answer sheets. 

Because ascertainment of relationships between the new meas- 
ures and widely used standardized tests of verbal ability, reading, 
and achievement was desired, the STEP Reading, Form B and the 
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SCAT, Form B were administered to the sixth-grade sample. 
STEP Reading was designed to measure ability to reproduce ideas, 
to translate ideas, to make inferences, to analyze motivation and 
presentation, and to criticize. SCAT was developed to measure 
sentence understanding, numerical computation, word meanings, and 
numerical problem-solving. 


Results 


Interrelationships of the Experimental Measures of Qualitative 
Styles 


Inspection of the correlational matrices in Table 1 showed the 
following results, The first hypothesis proposing that measures of 
each of the qualities—abstract, functional, and concrete—would 
be positively and substantially related across the other experimental 
measures within the (intended) same qualitative level was sup- 
ported partially. The correlations between abstract subscores 
ranged from .41 to .72. The functional subscores, however, showed 
no significant relationship with one another. The correlations varied 
from .02 to .29. At the third-grade level, the concrete subscores 
showed insignificant interrelationships. However, there were some- 
what stronger interrelationships at the sixth-grade level with one 
coefficient as high as .50. 

Why might these results have taken place? Possibly the rea- 
son for the consistently high correspondence of the abstract qual- 
ity from measure to measure rests in the basic nature of symbolic 
thinking—a quality or ability perhaps more encompassing and less 
anchored to the particular stimulus material than are the other two 
qualities, Accordingly, the variable called abstract quality in 
thinking—be it in problem solving associated with a story, in criti- 
cal and creative reading of paragraphs, or in the process of concept 
formation involving single stimulus words—appears to yield high 
positive correlations across the experimental measures. This cor- 
respondence was especially evident at the sixth-grade stage of de- 
velopment. For concrete responses, perhaps the problem-solving test 
and the word-meaning test (for which the intercorrelation was 
.42 for sixth-grade pupils) contained some identical elements in 
regard to Guilford’s products classified as units and classes. Or to 
phrase the comment another way, perhaps pupils who contributed 
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TABLE 1 
Validity Coefficients for Measures of Qualitative and 
Non-Qualitative Verbal Abilities 


Intercorrelation of Measures or Subtests 


N Grade Test Total Abstract Functional Concrete 


88 3 I&II 46 51 09 05 
I&II .45 Al 03 32 
TI & Il 27 43 11 OL 
86 6 I&II -76 .72 05 50 
I&I .62 .56 29 42 
I & II 64 62 06 38 
70 6 Critical Listen- 
ing and— 
I 55 AT 13 —.53 
II 66 62 07 —.47 
Til 67 .63 02 —.50 
70 6 Reading and— 
I 58 -50 11 —.49 
II .58 .59 —.04 —.48 
II 66 56 20 —.50 
70 6 SCAT and— 
I -60 53 -08 — .53 
I 57 -55 .05 — AT 
Ill .67 59 13 —.48 


Conop formation, (word teasing) tat, da ema” nt el Pararans tast, 48 items; I = 
to the concrete scores were operating at these more simple levels 
irrespective of whether the test afforded products at a more com- 
plex level. It should also be remembered that because of the 
method of scoring these ipsative measures where each answer 
choice in response to a question represents either the abstract, 
functional, concrete, or error quality, the magnitude of the correla- 
tions may be exaggerated because of experimental dependence in 


the responses. Consequently the correlational results must be 
viewed with some caution. 


Relationships of the Experimental Measures To Other Measures 


The second hypothesis concerning the relationship between the 
experimental Measures and three other tests (critical listening, 
reading, and scholastic ability) received some support. (See Table 
1.) The relationship between the problem-solving story and the 
creative-critical paragraph tests involving qualitative levels of 
thinking did indeed show a higher correlation (.76) than did these 
two measures when each was correlated instead with each of the 
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two standardized tests, reading and achievement, and with the test 
of critical listening. The correlations with the non-qualitative 
measures ranged from .55 to .67. Furthermore, although these cor- 
relations are substantial, their magnitude would not necessarily 
indicate that one measure could meaningfully be substituted for 
another, even in the case of tests involving problem solving with 
story content and creative-critical thinking in paragraph stimulus 
material. 

From examination of the intercorrelations of subscores in the ab- 
stract, functional, and concrete categories, it ‘was apparent that 
the main contribution to the positive substantial relationships was 
coming from the abstract quality. The correlations of the abstract 
subscores with the three non-qualitative measures varied from 
47 to .63 with most of the correlations falling in the .50 to .59 
range. It appears that children who read, achieve, and listen well 
appear to be able to operate with abstract responses. 


Developmental Differences 


The third hypothesis proposing differences between average 
scores of third- and sixth-grade pupils in qualities of thinking ap- 
peared to be supported. The results were in keeping with the theory 
stimulated by Piaget and represented in this country by Reichard, 
Schneider, and Rapaport (1944). Across all three qualitative 
measures there was significant dominance of the concrete choices 
for the third-grade children with considerable decline of concrete 
choices for pupils in the sixth grade. Consistent with the direction 
of the third general hypothesis, differences between the mean scores 
on the abstract quality and between mean scores on the concrete 
quality were statistically significant beyond the .01 level for the 
pupils in the third and sixth grades. (See Table 2.) 

Regarding the functional quality, it appeared that both grades 
shared this supposedly intermediate step between concrete and ab- 
stract thinking qualities. Similar findings were obtained for all 
three experimental tests. 


Impact of the Stimulus Material on Cognitive Style 


Examination of the data in regard to the fourth hypothesis gave 
some surprising results. Individual pupils were classified with re- 
spect to the predominant quality of their performance on each of 
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TABLE 2 


Significance of Differences between Test Scores Reflecting Categories of 
Choices Chosen as “Best” by 174 Children 


Grade3 N = 88 Grade6 N = 86 


Test Category M SD M sD t values 
1. Problem- 
Story 
Abstr. 10.77 4.08 19.01 5.28 11.52* 
Funct. 10.90 2.65 11.65 3.04 1.74 
Concer. 11.67 3.79 6.20 3.74 9.55* 
2. Creative 
& Critical 
Paragraph 


Abstr, 14.06 5.69 21.22 6.40 7.80* 


Funct. 12.42 3.73 11.57 3.27 1.60 
Coner. 15.79 21.57 8.80 3.95 2.96* 
8. Concept 
Formation 
(Word 
Meaning) 


Abstr. 14.75 5.78 23.25 6.09 9.49* 
Funct. 16.02 3.75 16.52 3.70 .88 
Concer. 11.54 4.39 5.64 3.52 9.78* 


* Significant at .01 level, ¢ = 2.60. 


the three experimental tests. Each child’s dominant preference for 
any one quality—abstract, functional, or concrete—was identi- 
fied as mentioned previously by an arbitrary criterion of at least 
40 per cent of total number of responses in that category and at 
least 10 per cent fewer of the total responses in any other category. 
The following results are given in percentages for 90 third-grade 
pupils and 88 sixth-grade pupils. Although only 24 per cent of the 
third-grade pupils exhibited a distinct cognitive style, the majority 
of this group demonstrated a concrete style on the word sitmulus 
test (18%). Nine per cent of the third grade pupils displayed an 
abstract style and two per cent displayed a functional style. Of 
those showing a dominant cognitive mode in the sixth grade, the 
majority exhibited an abstract style (67%). The rest did not reach 
the criterion. A higher proportion of sixth-grade pupils than of 
third-grade pupils did exhibit a style. 

Regarding the more complex materials, the story and the para- 
graph, the sixth-grade exhibition of style remained much the same. 
For the third grade, however, there was an interesting and surpris- 
ing trend. Instead of an increase in concrete cognitive style pre- 
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dicted for the more complex stimulus materials, third-grade 
pupils exhibited an increase in abstract style. As the stimulus ma- 
terials became more lengthy and complex in the paragraph and 
in the problem-solving story units, as the questions supposedly 
provoked more complex operations, it appeared that more, rather 
than fewer, third-grade pupils were tending to think with an 
abstract style. Perhaps the major factor was that the latter two 
experimental measures (problem-story test and creative and criti- 
cal paragraph test) were simply more intriguing and interesting 
than was the word stimulus test. For the concept formation 
(word stimulus) test 9 per cent of the third-grade styles fell into 
the concrete category. For the problem-story and the critical and 
creative paragraph test the percentage of abstract styles, at the 
third-grade level, increased to 21 per cent. An ingredient of affec- 
tive involvement stimulated by the more challenging material may 
have brought about, in part, this more than doubling of percentages, 
that is, from 9 per cent to 21 per cent. 


Results from a Factor Analysis 


Finally, although the details of the factor analysis are not pre- 
sented because of the limitations posed by experimental dependence 
among the measures associated with ipsative properties in scoring, 
a few general findings suggested some possible noteworthy rela- 
tionships consistent with the previous correlational findings. 

Although the variables associated with subscores on concrete 
qualities from the problem-story test and the paragraph test 
appeared on the same factor at sixth-grade level, the concrete 
subscore from the concept formation (word stimulus) test was not 
related to this factor. In effect, all functional variables and all con- 
crete variables did not appear on one functional factor or on one 
concrete factor at the sixth-grade level, but two of the more com- 
plex concrete variables were related to the same factor. The rela- 
tionship of these two complex concrete variables is consonant with 
the assumption that creative and critical thinking processes are 
interrelated within the broad process of problem solving. 

At the third-grade level, however, there appeared to be no inter- 
relation among operations at the concrete level. Indeed, there was 
a unique factor for each concrete subscore category in each of the 
experimental tests. However, loadings were present for two ab- 
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stract variables on a common factor at the third-grade level. Spe- 
cifically, both the problem-solving story test and the creative-critical 
paragraph test showed loadings on the same factor. From these 
results it may be inferred that if a third-grade pupil is capable 
of abstract thinking, his pattern of interrelationship might more 
closely approximate the factor structure of sixth-grade pupils or 
more mature students. 

Finally, with respect to the factor of concrete concept formation, 
the test measures—critical listening, STEP Reading, and SCAT— 
showed high negative loadings. This factorial result is consonant 
with the correlational findings. Stated negatively, it appears that 
children who tend to exhibit concrete qualities of concept formation 
do not do well on tests of critical listening, reading, and scholastic 
achievement, 


Conclusion 


A study was made to ascertain interrelationships at each of two 
grade levels between three experimental tests purporting to measure 
three qualitative levels of verbal functioning, a test of critical 
listening, STEP Reading, and SCAT. A theoretical framework 
stemming from early work by Piaget and set forth by Russell was 
the major basis for construction and validation of the tests. 

Results indicated that measures of verbal thinking with an ab- 
stract quality are closely related irrespective of whether the stimu- 
lus material is a word, a paragraph, or a story. Relationships, how- 
ever, do not appear high enough to justify the substitution of one 
process measure for another. Functional and concrete qualities of 
thinking are not related significantly either to the measure of listen- 
ing or to standardized tests of reading and general verbal ability 
used in the present investigation. Developmentally, correlational 
data for each of the new or revised measures seem to sustain the 
theoretical framework from earlier investigations. There was a sig- 
nificant difference (.01 level) between the means for the third- and 
sixth-grade group on the abstract and concrete scores for all three 
measures. Irrespective of which of three experimental tests was 
employed, the third-grade means were substantially higher on the 
measures of the conerete quality than on the measures of the other 
two qualities. In contrast, sixth-grade means were noticeably 
higher on the measures of abstract quality than on the concrete or 
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functional qualities. Lastly, increase in complexity of verbal stimu- 
lus material, however, appeared to stimulate a preference for ab- 
stract cognitive style even at the third-grade level. The appearance 
of this increase should be provocative of further research in test de- 
velopment at the third-grade level, in which specific training and 
carefully devised materials might be used to stimulate additional 
abstract thinking in younger children. 
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Tun Reading Prognosis Test (RPT) was constructed to provide 
adequate measures of skills underlying reading of children in both 
middle and lower socio-economic levels. It consists of seven sub- 
tests, grouped in three areas. 

In the Beginning Reading area are tests of Alphabet Letters and 
Sight Vocabulary. The Perceptual Discrimination area includes 
tests of Auditory Discrimination, Visual Similarities, and Visual 
Discrimination. In the Language area are tests of Meaning Vo- 
cabulary and Storytelling. 

Previous validation studies with the RPT indicated that it was a 
good overall predictor of reading achievement as measured by 
standardized reading achievement tests administered at the end of 
the first or second grade (Weiner and Feldmann, 1963). The present » 
study was undertaken during the school year 1963-1964 in order to 
validate a revised form of the RPT. 


Test Item Changes from Third to Fourth Study 


Since the third validation study indicated a need to revise cer- 
tain items and directions, the subsequent revision of the RPT in- 
cluded the changes described below. 

1 Consultants: Institute for Developmental Studies, Department of Psy- 
chiatry, New York Medical College. 
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First, a new Meaning Vocabulary subtest was constructed 
through using a completely different set of words. The new words 
were selected on the basis of frequency of use by different grade 
levels as listed in the Rinsland Word List (1945), and because of 
their statistical comparability to the words formerly used. 

Next, because in the three previous studies the items of the Audi- 
tory Discrimination Test showed little discrimination power, a list 
of 26 items, most of them new items consisting mainly of nonsense 
words, was introduced. Another problem with the Auditory Dis- 
crimination Test was in the clarity of its directions. They were 
changed to include a clearer statement to the respondent of what is 
meant by “same” and “not the same,” a concept which children of 
lower socio-economic level have found difficult. This subtest was re- 
tained despite its difficulties because previous studies at the Insti- 
tute for Developmental Studies had found positive results with an 
Auditory Discrimination Test in relation to reading achievement 
(Katz and Deutsch, 1963). 

The remaining subtests, Sight Vocabulary, Alphabet Letters, 
Storytelling, Visual Discrimination, Visual Similarities, and Be- 
ginning Reading, did not require revision. However, general direc- 
tions for all tests, directions to be read to the pupils, and scoring 
procedures were revised. 


Procedures 


For the present study, a new form of the Institute for Develop- 
mental Studies’ Socio-Economic Classification Scale was used to 
determine Socio-Economic Status (SES) of the subjects. In the new 
scale the children were classified into three SES levels, with the low 
group as SES I and the high group as SES III. Only two classifica- 
tions had been used in earlier studies, 

One advantage of the three-level categorization is that the differ- 
cape between Levels I and III seems more definitive than previ- 
ously. 

‘The subjects were both drawn from selected schools in New York 
City and included the entire first grade population in a small West- 
chester community. All the children were tested with the RPT dur- 
ing the first six weeks of the first grade. At the end of the school 
year, either the Gates Primary Reading Tests (Paragraph Read- 
ing) or two parts of the Metropolitan Achievement, Tests Primary 
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I Battery Word Knowledge, and Primary I Reading were admin- 
istered to the subjects. 

Tables 1-6 report the intercorrelation coefficients among Reading 
Prognosis subtest area, total test scores, and criterion scores, 
grouped according to socio-economic levels and community. As one 
might expect, the Beginning Reading area appears to be the best 
predictor of end-of-first-grade reading ability for all groups. The 
Perceptual Discrimination area is the next best predictor, whereas 
the Language area yields the lowest r’s with the criterion measures. 

It appears that the Language and Perceptual Discrimination 
areas are less powerful predictors of reading achievement because 
of the presence of one subtest within each area respectively, Story- 
telling in the Language area and Auditory Discrimination in the 
Perceptual Discrimination area. Further analysis of the Storytell- 
ing scores revealed that of the 21 subjects who earned a raw score of 
3 or less, 16 earned a 1.61 or lower grade equivalent on the Metro- 
politan Achievement Primary I Reading Test. Analysis of the Audi- 
tory Discrimination subtest showed that of those 30 children who 
earned raw scores below 8, 18 earned a grade score of 1.6 or lower 
on the criterion achievement test. 

For the sample for whom the Gates Primary Reading Tests Para- 
graph Reading served as the criterion measure, the same relation- 
ship was found between low scores on the two RPT subtests and 
low scores on the criterion test. Although high scores on these sub- 
tests may be rather meaningless, low scores may prove useful for 
diagnostic purposes. 


Discussion 


It is evident from Tables 1-6, that the total score of the RPT is a 
good predictor of reading as measured by two different standard- 
ized reading achievement tests for children from different commu- 
nities and different socio-economic levels. Total scores obtained at 
the end of kindergarten or at the beginning of the first grade would 
appear to be useful for purposes of grouping for instruction as well 
as for anticipating future reading levels. 

The area scores, Beginning Reading, Perceptual Discrimination, 
and Language, are also good predictors of reading achievement. In 
spite of some problems with two subtests, Auditory Discrimina- 
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TABLE 4 


Intercorrelations of Reading Prognosis Subtest Area, 
Total Test, and Criterion Test Scores 
(Community B, SES I, N = 19) 


GPPR GPPR 
BR L PD TOT RS Gs 
1. Beginning Reading (BR) .340 .704 .879 817 .816 
2. Language (L) 246 6.560 294 +322 
3. Perceptual Discrimination (PD) 896 .738 .739 
4. Total Score (TOT) .811 .820 
5. Gates Primary Reading Tests 


(GPPRRS) Paragraph 

Reading-Raw Score 992 
6. Gates Primary Reading Tests 

(GPPRGS) Paragraph 

Reading-Grade Score 


Mean 4.11 8.00 15.31 27.42 6.36 1.75 
5.D. 3.74 2.77 518 948 5.87 45 
TABLE 5 


Intercorrelations of Reading Prognosis Subtest Area, 
Total Test, and Criterion Test Scores 
(Community B, SES II, N = 11) 


GPPR GPPR 
BR L PD TOT RS GS 
1. Beginning Reading (BR) .123 .760 885 -750 638 
2. Language (L) 539 523 —.144 —.308 
8. Perceptual Discrimina- 
tion (PD) 956 .573 361 
4, Total Score (TOT) 612 A27 
5. Gates Primary Reading 
Tests (GPPRRS) 
Paragraph Reading- 
Raw Score 935 
6. Gates Primary Reading 
Tests (GPPRGS) 
Paragraph Reading- 
Grade Score $ 
Mean 13.54 12.63 23.18 49.27 16.73 2.59 
8.D. 5.87 242 5.39 11.63 5.99 58 


tion and Storytelling, subtest scores as well as total area score 
should be valuable as diagnostic measures of pre-reading skills. 

The present study also indicated that another value of the RPT 
test is its individual administration. The teachers who had ad- 
ministered the test at the beginning of the school year had an 
opportunity to learn how their pupils respond to certain questions 
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TABLE 6 


Intercorrelations of Reading Prognosis Subtest Area, 
Total Test, and Criterion Test Scores 
(Community B, SES III, N = 18) 


GPPR GPPR 
BR L PD TOT RS GS 


A eer aaa a 
1. Beginning Reading (BR) 443 «780  .900 .712 659 
2. Language (L) ie 442 719 515 SCL. 
3. Perceptual Discrimina- 
tion (PD) 895 .800 666 
4. Total Score (TOT) 810 696 
5. Gates Primary Readin, 
Tests (GPPRRS) 
Paragraph Reading- 
Raw Score 922 
6. Gates Primary Reading 
Tests (GPPRGS) 
Paragraph Reading- 
Grade Score 
Mean 14.72 1211 24.00 50.83 15.00 2.55 
S.D. 5.69 4.65 5.43 13.33 7.00 85 
ene mete. FOU OO 


in areas related to reading. Determination of underlying skills with 
a standardized procedure should add to the teacher’s understanding 
of each pupil as well as to the teacher’s implementation of the 
curriculum. 


The RPT used in the present validation study is still undergoing 
minor revisions based on item analyses of the subtests, After pilot 


studies of these revisions, the test will be ready for standardiza- 
tion, 
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TE present investigation was designed as a construct validity 
study of the Duncan Personality Integration Scale (1963). Recent 
developments in clinical psychology and personality theory have 
emphasized the importance of conceptualizing and investigating 
integrated behavior or positive mental health (Wishner, 1955; Sho- 
ben, 1957; Seeman, 1963). This emphasis has created a strong 
need for objective and quantifiable instruments to measure such be- 
havior. Tests by Duncan (1963) and Fitts (1965) are apparently 
the only psychometric devices which are available for this purpose. 
These tests require immediate investigation of their validity so that 
they might either be employed to assess integrated behavior in both 
clinical and research situations, or be discarded in order that the 
search for more adequate measures may be begun. 

Items in the Duncan Seale derive content validity by virtue of 
the fact that they sample the six types of behavior which, according 
to a review by Jahoda (1958) of personality theory and research in 
the area, encompass the area of positive mental health. Specifically, 
the Duncan scale was designed to assist in researching Seeman’s 
(1959) theory of organismic integration. 

This instrument (see Table 1) requires subjects to nominate peers 
as being one of the three members of their group who is best de- 
scribed by a given item, such as: “Who are the three persons who 
seem best able to deal effectively with everyday tensions and anx- 
ieties?” Early attempts at construct validation of the scale have 
produced positive results. For instance, in one study (Duncan, 
1964), integrated subjects, in contrast to those who were less well 
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integrated, showed more positive self concepts, greater internal 
locus of control, more extra-curricular involvement, and a higher 
grade point average even though they did not possess high aptitude 
scores. This additional validity study sought to examine the rela- 
tionship of the Duncan Scale to response styles, to self concept, and 
to the reported perception of others as measured by a sentence 
completion technique described by Getzels (1951). 

‘The sentence completion task was a paired direct-projective ques- 
tionnaire (PGPQ) consisting of two sets of 95 open-ended stems, 
which differed only in that one set was presented in the first person 
and the other in third person form. Seventeen of the 95 items were 
personal in nature, e.g., “after I’ve finished talking, I usually feel 
that. . . .” The remaining 78 items were political items, e.g., “Helen 
feels that governmental rent controls should be... ,” which served 
as fillers, 

Getzels (1951) hypothesized that when subjects respond to third 
person items under speeded administration conditions, they tend to 
express personal feelings, since these are readily available to them. 
The third person form supposedly prevents subjects from recog- 
nizing that personal feelings are being expressed, and thus elicits 
material which, compared with that in the first person, is less 
socially desirable and less a part of conscious experience. Responses 
to first person stems are felt by Getzels to reflect feelings which are 
more socially desirable and which are more a part of conscious ex- 
perience than are those responses to third person stems, Theoreti- 
cally, then, the 17 personal items, when completed in both first and 
third forms, should provide measures of self concept (positivism or 
negativism in first person responses), reported perception of others 
(positivism or negativism in third person Tesponses), congruence of 
Tesponses (presence or absence and magnitude of discrepancies be- 
tween first and third person responses for the same item), and de- 
gree of commitment (Rubin-Rabson, 1954) in response set (the 
extent or degree of positivism or negativism employed in responding 
to a PDPQ item). 

Method 


Subjects 


The subjects were 49 single white female undergraduates at a 
large midwestern university. Each Subject resided in one of 4 units 
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which housed 14 to 16 sorority members. Subjects had been allowed 
to choose a roommate; however, mates were randomly assigned to 
units. 


Procedure 


Testing was done in two one-hour sessions which were separated 
by one week. During the first session, subjects were administered 
Form A of the PDPQ task. This form presented seven of the per- 
sonal items and 38 of the fillers in first person form, with the re- 
maining personal and filler items given in the third person. The 
following instructions were to read to provide a speeded condition: 

This is a test of how fast you can think. Complete each of the 
following sentences so that it makes the best sense possible. You 
may use either a word or a phrase, although a phrase is prefer- 
able. Since you will have only a limited time, work quickly. In 
most cases the best way to answer the test is to put down the 
first thing that comes to your mind. Example: The unfinished 
sentence: When Ann scored the goal, she . . . may be finished by 
adding “felt good” or “won the game” or “told her friend” or 
“wanted to score another” or something like that. This is a 
speed test. Work fast. Do not skip any sentences. 

At the second session, subjects completed Form B of the PDPQ 
which consisted of the same 95 items as Form A, but with the per- 
son of each stem switched from first to third or vice versa. The same 
speeded instructions utilized with Form A were employed. After 
completing Form B, subjects rated the other residents of their own 
housing unit by means of the Duncan Scale. 

Six of the subjects who were present at the first testing did not 
participate in the second session. Statistical analyses were per- 
formed on the data provided by the remaining 43 subjects who 
completed the entire battery. Scores on the Duncan Scale were 
corrected to allow for the number of residents from a given unit 
participating in the study, since the subjects’ scores depend upon 
the number of other subjects rating them. 

Each of the PDPQ responses was scored for positivism- 
negativism by two raters on a 1-5 Likert-type scale. The raters 
were naive in terms of previous PDPQ rating experience and also as 
far as knowledge of subjects’ scores on the integration scale. Inter- 
rater reliability was estimated by a Spearman rank-order correla- 
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tion between the two raters’ scores for the 95 first person and 95 
third person items provided by a single subject. Each of the Dun- 
can Scale items, as well as the total score, was correlated by means 
of a Pearson product-moment solution with each of the following 
three PDPQ variables: positiveness of all first person responses, 
positiveness of all third person responses, and sum of discrepancies 
between all first and third person ratings. 

In order to determine extent of commitment in response set, a 
Pearson product-moment correlation was estimated between inte- 
gration scores and the extent to which subjects’ responses for all 
items deviated from the neutral central tendency on the 1-5 Likert 
scale for positivism-negativism. 


Results 


Inter-rater reliability coefficient for the PDPQ was estimated to 
be .92. Correlations between the Duncan Scale items and the three 
PDPQ variables are shown in Table 1. Ten of these correlations 
were significant at a level less than .05. Only one such correlation 
would be expected by chance. 

As predicted, the Duncan Scale correlated with positiveness of 
the first person ratings on the PDPQ. Three of the six scale items, 
as well as the total score, correlated positively and significantly. 
The other three integration items correlated positively though not 
significantly. 

Also, as predicted, the Duncan Scale correlated with positiveness 
of third person ratings on the PDPQ. Two of the items, in addition 
to the total score, correlated positively and significantly with this 
variable. The other four items correlated positively though not 
significantly with this variable. : 

Contrary to prediction, item three of the Duncan Scale correlated 
significantly with the sum of all PDPQ first and third person dis- 
crepancies. Also, contrary to prediction, all five remaining items 
correlated positively, though not significantly with this variable, 

It will be noted that all six integration variables correlated in a 
positive direction with each of the three PDPQ variables. The 
probability of all six integration variables correlating in the same 
direction with a variable by chance, as measured by a two tailed 
sign test (Siegel, 1956, p. 68) is .032. 

The correlation between subjects’ integration scores and their 
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TABLE 1 
Correlation of Seven Duncan Scale Items with Three PDPQ Variables 
Sum of Dis- 
crepancies 
Positiveness Positiveness Between all 
of all First ofall Third First and 
Duncan Scale Person Person Third Person 
Items Responses Responses Ratings 


1. Who are the three persons in your 

unit who seem best able to ex- 

press their feelings without hurt- 

ing the feelings of others? .18 .35* 
2. In your opinion who are the 

three persons in your unit who 

seem to understand themselves 

best; that is, are aware of their 

shortcomings and strengths? .33* .34* 
3. Who are the ones in your unit 

who seem best able to keep an 

open mind and not jump to pre- 

mature conclusions? .34* 13 
4, Who are the three persons in 

your unit who seem the most 

able to deal effectively with 

everyday tensions and anxieties? 25 14 
5. Which three persons in your unit 

seem capable of forming deeper 

and more profound relationships 

with others and seem to be 

genuinely concerned with other 

people? ek ll 
6. Which three persons in your unit 

seem to you to have been the 


most successful in all phases of 

their life: social, personal, edu- 

cational, etc.? .31* +24 
7. Total Score .33* .35* 

*p <05 


G 


14 


.37* 


+09 


tendency to make positive or negative responses which deviate from 
the neutral central tendency was designed to relate integration to 
commitment in response set. This correlation was .38, which is 


significant at the .05 level. 


Discussion 


Performance on the Duncan Scale appears to be related to a posi- 

tive self concept. This notion is supported by the fact that three 

a Duncan scale items, as well as the total score, correlated positively 
: and significantly with the positive quality of the first person ratings 
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on the PDPQ. These data support earlier findings and contribute 
to the construct validity of the Duncan Scale. 

Ratings on the Duncan Scale were also related to a more posi- 
tively reported perception of others. This is indicated by the fact 
that two of the integration items, as well as the total score, corre- 
lated positively and significantly with the positive quality of the 
third person ratings. This is consistent with Rogerian self theory 
which assumes that the person who accepts himself will be, because 
of this self-acceptance, more accepting of others (Rogers, 1951, 
p. 520). This finding is also felt to contribute to the construct valid- 
ity of the Duncan Scale. 

There was a positive relationship between the Duncan Scale and 
the sum of discrepancies on the PDPQ. If this result is interpreted 
according to Getzels’ hypothesis, it suggests that integrated sub- 
jects are less congruent in their response style than are noninte- 
grated subjects. This conclusion coupled with results of a study by 
Getzels (1951) which utilized normal and disturbed subjects would 
suggest that congruence, as measured by a PDPQ type instrument, 
is curyilinearly related to integrated behavior. However, the fact 
that persons scoring high on the Duncan Scale tended to have more 
discrepancies in first and third person responses on the PDPQ can 
be explained in part by their tendency to make more committed re- 
sponses. Subjects scoring low on the Duncan Scale made more 
neutral responses, and neutral responses produce fewer discrepan- 
cies. Also, the discrepancies of subjects who scored high on the 
Duncan Scale were not the result of their completing the first per- 
son items with positive statements and the third person items with 
negative statements as Getzels’ disturbed subjects had done. The 
higher sum of discrepancies for integrated subjects resulted from 
the fact that they possessed the greater tendency not only to rate 
themselves higher than others, but also to rate others higher than 
themselves. In this case, the word “openness” seems to describe 
better their behavior. 

This finding raises the possibility that integrated subjects were 
less likely than disturbed subjects to project personal feelings in re- 
sponding to third person stems. If so, the third person stems, in 
contrast to first person stems, elicit responses which are more indic- 
ative of the integrated subjects’true perceptions of others. If this 
hypothesis is true, it points up a methodological problem that must 
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be considered in research on personality integration: constructs may 
need to be operationally defined in a manner for effective subjects 
different from that for normal or disturbed ones. 


Summary 


Forty-three single white female undergraduate sorority members 
at a large midwestern university were administered the Duncan 
Scale, a measure of personality integration, and a projective-direct 
personality questionnaire (PDPQ). The latter scale yielded scores 
for commitment and congruence in response styles, self concept, and 
reported perceptions of others. Ten of the 21 correlations between 
the integration scale and PDPQ variables were significant. It was 
concluded that performance on the Duncan Scale is related to a 
relatively high positive concept of both self and others, as well as 
to a tendency to make committed, as opposed to non-commital, 
responses. These findings were interpreted as supporting the con- 
struct validity of the Duncan Scale. Reference was made to a 
methodological problem concerning the operational definition of 
constructs for effective subjects. 
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À IDENTIFICATION OF FOUR ENVIRONMENTAL PRESS 
id FACTORS IN THE STERN HIGH SCHOOL 
CHARACTERISTICS INDEX 


HOWARD R. KIGHT 
AND 
EDWIN L. HERR 
State University of New York, Buffalo 


A 


j Problem 


hy 
f 

! Tum purpose of this investigation was to ascertain the factored 
by dimensions of the 30 subscales on the Stern High School Character- 
istics Index (HSCI). The HSCI is one of four instruments devel- 
oped by Stern, Stein, and Bloom (1956) to measure environmental 
q press variables but is the only one specifically designed for the high 
i f school environment. Wolf (1965) has recently called attention to 
¿the growing importance of assessments of this type and the need for | 
es validating research evidence. 


; i Procedure 


Two samples of 725 and 348 students from two different high 
schools were administered the HSCI. A principal components anal- 
i ysis was performed from use of the 30 subscale scores obtained for 
each sample and then rotated analytically according to Kaiser’s 
(1960) varimax criterion. 


ma 


Results 


Although six factors were extracted with the first sample (A) and ; 

% five factors from the second (B), only the first four factors in each 

sample, which appeared to be invariant, could be interpreted mean- 
ingfully. They were identified as: 
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ey 


1. Social-intellectual. avoidance—a tendency to’ avoid or with- 
draw from intellectual and/or social situations which might be 
competitive or lead to frustration, humiliation, and a loss of self- 
esteem, (Factor I) > 

2. Inferiority reaction—a tendency to counteract criticism or 
feelings of inferiority by becoming indifferent to or disregarding 
the opinions and feelings of others. May also be manifested into 
overt forms of behavior. (Factor II) 


TABLE 1 


Factor Loadings of Thirty Subscales on the Stern High School 
Characteristics Index For Two Samples 
(Na = 725, Nx = 348) 


a Factors ‘ 
EE EnEEn Nn 
I MIL Til IV 
Subscales A B A B A B AAB 
1. Abasement 25 34 73 72 —05 04 22 —05 
2. Achievement —60 —56 -14 -15 20 06 —20 20 
3. Adaptability 04 07 69 65 10 11 02 29 
4. Affiliation —40 —67 —26 -20 03 —09 —62 26 
5. Aggression 17 20 64 71 4-388 —25 12 07 
6. Change ol —31 ol 22 =11 —29 —09 —il 
7. Conjunctivity —43 —49 —32, —49 37 10 —32 30 
8. Counteraction —26 —59 —13 -10 15 00 —06 06 
9. Deference -04 04 Ql -17 60 + 23 02 04 
10. Dominance 10 04 58 49 —04 —09 —23 56 
11. Ego-Achieve- 
ment -63 —75 —04 -08 15 —05 —21 —06 
12. Emotionality -18 —26 04 04 —04 —44 —03 26 
13. Energy ` —58 —63 © —25 -28 18 —08 —32 17 
14. Exhibitionism —47 —63 04 -13 05. —12 -53 39 
15, Fantasied R 
‘Achievement —51 —24 % 07 11 —17 —02 21 —04 
16. Harm Avoid- ` 
ance =32 —22 . —36 —20 42 25 08 —04 
17. Humanism -76 —75 —09 -10 13 19 —04 —07 
18. Impulsiveness 04 22 23 —01 —38 —70 -17 20 
19. Narcissism —29 —47 —03 -28 * 29 13 -44 53 
20. Nurturance —59 —60 —19 -25 23. #15 - =2"7 13 
21. Objectivity —35 -32 —68 -77 w cor i6 O01 
22. Order -22 —21 —03 -11 64 49 -17 4 
23. Play $ —15 _ —42 —13 =22 —13 -38 -74 36 
24, Practicalness —07 —24 14 05 32 —12 —42 63 
25. Reflectiveness —73 —76 —08 -12 13 03 -17 07 
26. Scientism —68 —58 —20 -24 13 0% -ll 11 
27. Sensuality —55 —48 —31 -28 -0 05 -1il —0l 
28. Sexuality 07 —02 34 2 —09 —16 -45 73 
29. Succorance —51 —29 —43 -47 06 —17 —08 07 


30. Understanding —60 —56 —30 -28 25 07 -15 02 
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3. Compulsivity and restraint—a tendency to be cautious, orderly, 
and reflective in conforming to social and intellectual demands. 
(Factor IIT) 

4. Heterosexual dominance—a tendency to desire the athletic, so- 
cial, and intellectual qualities for heterosexual dominance or popu- 
larity. (Factor IV) 
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PREDICTING GRADUATE SUCCESS AT WINONA 
STATE COLLEGE 


CONSTANCE M. ECKHOFF 
Winona State College 


Problem 


The purpose of this study was to investigate the relationship be- 
tween selected background variables and achievement in graduate 
courses at Winona State College and to determine how accurately 
these variables predict achievement. 


Background Variables 


The background variables selected for investigation were (1) un- 
dergraduate grade point average (UGPA), (2) Miller Analogies 
Test scores (MAT), and (3) Advanced Education Section (AE) 
scores from the Graduate Record Examinations. 


Criterion 
The criterion was overall graduate grade point average (GGPA) 


in which grade points were assigned to marks as follows: A = 3, 
B=2,C=1,D=0, and E or F = ~—1, 


Statistical Methods 


Stepwise multiple regression analysis was used to investigate the 
relationship among the background variables and the criterion. 
With this method, the matrix of correlations among predictor vari- 
ables was inverted, and beta weights for each of these variables 
were calculated. A ‘t’ test was used to test the significance of each 
beta weight against zero at the .05 level. The predictor variable that 
yielded the beta weight with the smallest ‘t’ value was deleted from 
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the inverse of the matrix at each step until all weights reached sig- 
nificance. Beta weights were then calculated for the remaining pre- 
dictor variables through use of the new inverse. 


Subjects 


The subjects were 185 secondary education majors and 111 ele- 
mentary education majors with thirty or more quarter hours ac- 
cumulated at Winona State College. 


Results 


Tables 1 and 2 present the results of stepwise multiple regression 
analysis for secondary and elementary majors, respectively. Two 
variables, UGPA and MAT, yielded significant regression weights 
for predicting GGPA for secondary majors. This optimum regres- 
sion function yielded a multiple correlation coefficient of .51. Thus, 
although the content of the Advanced Education Section does appear 
relevant to graduate work in education, the test did not have a high 
degree of relationship with graduate achievement. When this varia- 
ble was deleted after the first step of analysis, no great decrease in 
the multiple correlation coefficient resulted, For elementary majors, 
two variables, UGPA and AE, yielded significant regression weights 
for predicting GGPA. This optimum regression function yielded a 
multiple correlation coefficient of .30. The MAT, which did not 
yield a significant degree of relationship with graduate success for 
these subjects, was deleted after the first step of analysis. No de- 
crease in the multiple correlation coefficient occurred. 


Conclusions 


This investigation of the relationship of selected background vari- 
ables to graduate grade point average suggested the following con- 
clusions: 

1. Optimum prediction of graduate success for secondary educa- 
tion majors can be made by using a least-squares regression func- 
tion containing background variables of undergraduate grade point 
average and a score on the Miller Analogies Test. 

2. Optimum prediction of graduate success for elementary educa- 
tion majors can be made by using a least-squares regression function 
containing background variables of undergraduate grade point av- 
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erage and a score on the Advanced Education Section of the Gradu- 
ate Record Examinations. 

It is recognized that additional cross-validation studies should be 
undertaken with new groups of students. Such investigations have 
been planned for future classes of students. 


TABLE 1 
Beta Weights and Multiple Correlation Coefficients Before and 
After Deletion of Non-Significant Variables for 
Secondary Majors. (N = 185) 


Variable 
Statistic AE UGPA MAT Ri.2a..j 
Beta -1133 4053 -1639 52 
Beta -4070* -2335* 51 
*p <.01 
TABLE 2 


Beta Weights and Multiple Correlation Coefficients Before and 
After Deletion of Non-Significant Variables for 
Elementary Majors. (N = 111) 


Variable 
Statistic MAT UGPA AE Risa) 
Beta — .0007 .2142 .1811 -30 
Beta .2141* .1807** -30 
a 
*p <.01 
"p <05 
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PREDICTING ACADEMIC PERFORMANCE 
IN A SMALL SOUTHERN COLLEGE 
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Purpose 


The purpose of this research was to study the validity of the 
School and College Ability Test (SCAT) in predicting academic 
performance at Louisiana College, Pineville, Louisiana. 


Predictor 


The SCAT (Form 1A) was the predictor variable studied. Con- 
structed by Educational Testing Service, the SCAT is an aptitude 
test designed to aid in estimating the first year college applicant’s 
ability to perform college level work. The test, which measures 
school-related abilities of a verbal and quantitative nature, yields 
three scores: (1) Verbal, (2) Quantitative and (3) Total. 


Criterion 
Grade point average served as the criterion in the study. Both 


first-year grade point average and overall four-year grade point av- 
erage were employed as measures of academic performance. 


1 Full-time staff psychologist at Central Louisiana State Hospital and part- 
time assistant professor of psychology at Louisiana College. 
2 Now at the University of North Carolina, Greensboro, North Carolina. 
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Subjects 


The SCAT was administered to 698 entering college freshmen 
and covered a five year period. 


Results and Discussion 


Pearson product-moment correlations between first-year grade 
point average and the Verbal, Quantitative, and Total scale scores 
were .48, .16, and .48, respectively (N = 698). The Verbal and 
Total scales were found to be significantly correlated at the .01 level, 
but the correlation for the Quantitative scale was not statistically 
significant. 

SCAT scores were correlated with each student’s four-year grade 
point average with scores available on 110 students. Pearson prod- 
uct-moment correlations for the Verbal, Quantitative, and Total 
scales were .68, .38, and .61, respectively. Although the correlations 
were all statistically significant, the Verbal scale was the best pre- 
dictor of four-year grade point average. 

Several prior studies have disclosed significant positive correla- 
tions between all three SCAT scores and grade point average 
(Mann, 1961; Vick and Hornaday, 1962; Lewis, 1962). However, 
there have been conflicting results with the Quantitative scale, For 
example, Boyce (1964) reported that the SCAT Quantitative score 
Was not a significant predictor of college mathematics grades. Endler 
and Steinberg (1962) found that all three SCAT scales were sig- 
nificantly correlated with the first year grade point average correla- 
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EFFECTS OF ANSWER-SHEET FORMAT 
ON ARITHMETIC TEST SCORES 


HENRY F. DIZNEY 
PHILIP R. MERRIFIELD 
AND 
0, L. DAVIS, JR. 
Kent State University 


LınpouIsT (1964) has suggested two general criteria against 
which the design of answer sheets for tests may be judged: 

1. The design should minimize the effect of irrelevant factors on 
test scores, i.e., mechanics of answer sheet use must not in- 
fluence the test’s validity ; and 

2. Within the limits of the first criterion, the answer sheet should 
be as convenient to administer and mark, and as economical 
to use as possible. 

Lindquist properly emphasized the first criterion. Its implications 
would naturally vary with the nature of the examinees and scoring 
system as well as with the purpose of the test. Despite the obvious 
importance of the suggested criteria, the widespread use of separate 
answer sheets, and various methods of scoring, little empirical evi- 
dence is available concerning the effects of answer sheet format 
upon test results or examinees. One of the few studies along these 
lines was completed recently by Miller (1965). He investigated the 
effects of altering both color and marking position of answer sheets 
upon Lorge-Thorndike Intelligence Test scores. He studied these ef- 
fects at three grade levels in four cities and concluded that both 
color and answer format had non-significant effects upon test re- 
sults for eighth and twelfth graders. For fourth graders, significant 
interaction effects prevented a neatly drawn interpretation. Clearly, 
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theoretical and practical questions persist about the effects on differ- 
ent kinds of tests of varying answer sheet formats. 

The present study was conducted in contemplation of a change 
from IBM 805 to IBM 1230 scoring systems in a university testing 
center. The two scoring systems require different answer sheet for- 
mats, the 805 being generally well-known by most students. The 
805 format has response blocks numbered in columns from top to 
bottom of the page; the 1230 format, new to most students at this 
time, has answer blocks numbered in rows from left to right. 


Procedure 


It was decided to compare the performance of students using each 
format on the same test. The test chosen was one routinely admin- 
istered to all sophomores in the College of Education as an arithme- 
tic proficiency test. Failure on the test necessitates enrolling in a 
non-credit remedial course designed to alleviate the deficiency. Us- 
ing this test seemed to insure two desirable conditions: (1) an un- 
selected group; and (2) one whose members were motivated to 
perform well. The problems in the test were arrayed in columns and 
numbered from top to bottom; the 805 format was, in a sense, 
favored by this arrangement of items, 

A brief questionnaire was designed to elicit affective responses con- 
cerning the answer sheet used for the test. Copies of the two types 
of answer sheets were distributed at random along with the test 
booklets and questionnaires to 499 students presenting themselves 
for examination early in the fall quarter 1964. As a result of the 
exigencies of assigning students to testing groups, 240 responded to 
the 805-format answer sheet and 259 used the 1230-format answer 
sheet. It was believed that the assignment of answer sheets to stu- 
dents was essentially random with respect to all performance-related 
variables, and that the sample sizes were sufficiently large to miti- 
gate alarm. Questionnaire Tesponses and test scores were collated 
and analyzed, 


Results 


In response to each of three questions 
dents using the 1230 format reported d 
sheet than did those students using tl 
median test of the scores of those rep 


, proportionately more stu- 
ifficulty in using the answer 
he 805 format. However, a 
orting difficulty using the two 
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formats indicated no significant differences. In Table 1 the chi- 
square values for each question are presented; values in column A 
refer to frequency of reported difficulty, and are all significant at 
the .001 level; values in column B resulted from the median test 
applied to scores, and are not significant. 


TABLE 1 
Summary of Statistical Analysis 
Question As Be 
Difficulty in: x(df=1) x(df =1) 
a. using the answer sheet 89. 1.65 


b. orienting to the arrangement of 


answer blocks 142, .01 
c. coordinating problems and 
answer blocks 41, 2.90 


a x? value for frequency of users reporting difficulty in two format groups. 
b x? value, median test, for scores obtained by those reporting difficulty in two format groups. 


Students using the 1230 format very definitely felt greater diffi- 
culty in the testing situation. Nevertheless, in terms of their test 
performances, under conditions of high motivation, format mattered 


not. 
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SOCIO-ECONOMIC BACKGROUND AND FAILURE 
IN THE HIGH SCHOOL EXAMINATION 


8. L. CHOPRA 
Lucknow, India 


Onm of the serious problems facing educators in India is the rather 
high percentage of failures in the high school examinations. In the 
yearin which the present study was conducted 53.85 per cent of the 
high school examinees in Uttar Pradesh had failed. In the present 
study, therefore, an effort was made to see (Coster, 1959) how far 
these failures were related to socio-economic background and 
(Hohne, 1949) whether the differences in the percentages of failures 

+ in different socio-economic groups could be accounted for by varia- 
tions in levels of measured intelligence. 


Procedure 


The test of Progressive Matrices was administered to 1,359 ran- 
domly selected high school students (age range 14-17) studying in 
twenty-two urban and six rural secondary schools in Lucknow dis- 
trict. Data for parental occupation and results in the high school ex- 
amination were also collected. 


Results 


In Table 1 the entries show the percentages of failures in the dif- 
ferent occupational groups. It is apparent that the differences be- 
tween the number of failed students in the various occupational 
groups were statistically significant. Figures in column four show 
that there was a gradual rise in the percentage of failures as one 
moves from the higher to the lower occupational groups. 

Were the higher percentages of failures in the lower occupational 
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TABLE 1 
Percentages of Failures in the Different Occupational Groups 


Failures 
pki Number of 

Parental Occupation Students No. % 

(1) on (2) (3) (4) 
oS l 

1. Prof., Admn., Exec. & Managerial 186 50 27 

2. Clerical ‘ 268 138 51 

3. Minor Business & Sales Workers 226 124 55 

4, Skilled Workers 240 133 55 

5. Agriculturists 320 190 59 

6. Unskilled Workers 119 72 61 

Total 1,359 707 52 


XK? = 28.48 (p < .01) 


groups due to lower intellectual level of students in these groups? 
To answer this question, six groups of 81 students each, from the 
six occupational groups, were matched in intelligence test scores 
(person for person). In Table 2 the percentages of failures in these 


TABLE 2 
Percentages of Failures in the Different Occupational Groups 
Matched for Intelli 
Failures 
í Number of 
Parental Occupation Students No. % 
(1) (2) (3) (4) 
1, Prof., Admn., Exec. & Managerial 81 19 23 
2. Clerical 81 41 51 
3. Minor Business & Sales Workers 81 42 52 
4. Skilled Workers 81 46 57 
5. Agriculturists 81 49 60 
6. Unskilled Workers 81 49 60 
Total 486 51 
X? = 15.57 (p < 01) ne 
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six matched groups are shown. From the entries in Table 2 it ap- 


s of failures in the various 
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These results suggest that socio-economic background is positively 
related to success in the high school examination and that even when 
measured intelligence is held constant the richer the socio-economic 
background the less is the probability of failure. 
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PREDICTION OF GRADES IN GRADUATE 
EDUCATION COURSES 


L. L. AINSWORTH anp A. M. FOX 
Sam Houston State College 


Problem 


Tux purpose of this study was to determine the extent to which 
Miller Analogies Test (MAT) scores may be useful for the predic- 
tion of grade-point ratios (GPR’s) in graduate Education courses. 


Sample 


The sample consisted of 1,413 students at Sam Houston State 
College for whom MAT scores and at least one grade in a graduate 
Education course were available, for the period July, 1959, through 
August, 1964. 


Procedure 


GPR’s were calculated, and then relationships of MAT to 
GPR’s were determined by the Stepwise Regression Analysis Pro- 
gram (STRAP) in an IBM 1620 Data Processing System. An F- 
ratio of 4.0 was used for the criterion in testing the significance 
of the relationships. Significant relationships were found for MAT 
scores with total GPR (GPRT), basic required courses (GPRB), 
Principles of Guidance (GPR33), Measurement and Evaluation 
(GPR93), Secondary Curriculum (GPR94), and Human Growth 
and Development (GPR97). Regression equations, coefficients 
of alienation (k), and indices of forecasting efficiency (HZ) were 
then computed for these significant relationships. 


Results 
The results of this study are summarized in Table 1. 
499 
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TABLE 1 
Relationships of MAT to Dependent Variables 


Mean S.D. Equation 


GPRT 1,413 0.31 0.95 325 134 3.1 56 2.7 + 0.013(MAT) y 
GPRB 1,185 0.30 0.95 326 132 3.0 0.64 25 +0.015(MAT) 0.61 
GPR33 349 0.28 0.96 338 13.6 31 0.76 2.6 +0.016(MAT) 0.73 
GPR93 957 0.31 0.95 324 132 30 071 2.4 +0.017(MAT) 0.67 
GPR94 539 0.19 0.98 336 13.6 31 0.66 2.8 + 0.009(MAT) 
GPR9 640 0.25 0.97 328 133 30 0.68 2.6 +0.013(MAT) 


“ye 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
Vow. XXVI, No. 2, 1966 


PREDICTIVE RELATIONSHIPS BETWEEN ITEMS ON THE 
REVISED STANFORD-BINET INTELLIGENCE SCALE 
(SBIS), FORM L-M, AND TOTAL SCORES ON RAVEN’S 
PROGRESSIVE MATRICES (PM), BETWEEN ITEMS ON 
THE PM AND TOTAL SCORES ON THE SBIS, AND BE- 
TWEEN SELECTED ITEMS ON THE TWO 
SCALES? 


E. GEORGE SITKEL 
Pacific State Hospital 
AND 
WILLIAM B, MICHAEL 
University of California, Santa Barbara 


AurHoucH Raven’s Progressive Matrices (PM) test was origi- 
nally developed in England, it has been widely used in America for 
a number of years. Relatively little systematic research has been 
conducted concerning its relationship to well known scales of in- 
telligence developed in the United States. In his review of litera- 
ture pertinent to the PM, Burke (1958) noted that only a few 
studies had been concerned with the validity of the scale or with 
its item characteristics. Two other studies which reported correla- 
tions between scores on the PM and scores on parts of other in- 
telligence tests were those of McLeod and Rubin (1962) and 
Martin and Wiechers (1954). 


Purpose 


It was the purpose of the present study to report for separate 
samples of 63 men and 80 women as well as for the total sample 
1§upported in part by the National Institute of Mental Health Grant No. 
M4H08667: Socio-Behavioral Study Center for Mental Retardation, Pacific 


State Hospital, Pomona, California. Computing assistance was obtained from 
the Health Sciences Computing Facility, UCLA, sponsored by NIH Grant 
FR-3. 
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of 143 adults the following information: (1) a brief summary of 
the range and degree of the biserial coefficients of correlation of 
items at or above the eighth year level in the Revised Stanford- 
Binet Intelligence Scale (SBIS), Form L-M, with total scores in 
the PM, (2) a comparable summary of the biserial coefficients found 
between each item in the PM with total scores on the SBIS, and 
(3) a short interpretative statement concerning any pattern of rela- 
tionships associated with the 25 highest intercorrelations (phi co- 
efficients) found between any one of the items on the SBIS and 
any one of the items on the PM. The biserial coefficients of correla- 
tion were calculated on an IBM 7090 computer through use of the 
IBS-003 biserial and point biserial program at the Western Data 
Processing Center at UCLA. The phi coefficients were derived 
from the BIMED 02D program. 


Sample 


The total sample studied consisted of individuals ranging in 
age from 16 to 49 who were tested in their homes as participants in 
a community-wide survey concerned with identification of mentally 
retarded individuals in a city of about 100,000 population in South- 
ern California (Mercer, Dingman, and Tarjan, 1964). Of a pre- 
selected 10 per cent stratified sample of a total group of home 
dwellers in the community, interviewers had previously visited suc- 
cessfully 2,661 of the pre-selected households, a number which rep- 
resented about 91 per cent of the pre-selected group. During and 
subsequent to his visit in a household, the interviewer completed 
a questionnaire for each person and obtained information on his 
socio-developmental history. In order that a substantial range of 
ability level might be realized, two random samples were ob- 
tained: (1) one of persons who scored in the lowest 10 per cent on 
the socio-developmental scale, and (2) one of all persons in the 
interviewed households, 

These two samples were combined in such a manner as to ob- 
tain a distribution of scores in the two ability measures so that a 
proportional stratified sample would result, As defined by the 
selection procedures, based on the socio-developmental scale em- 
ployed in the community survey, a distribution of frequencies 
would probably be approximately Tectangular relative to the estab- 
lished percentile norms for the scale, Thus, with respect to an 
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existing normative C-scale of 10 steps for the socio-developmental 
measure the number of individuals selected for testing at high, 
medium, and low portions of this measure—steps 8, 9, and 10 for 
high; steps 4, 5, 6, and 7 for medium; and steps 1, 2, and 3 for low— 
were 59, 89, and 64, respectively. Actually tested at each of these 
levels were 50, 70, and 48 individuals, respectively. In relation to 
intelligence, the actual distribution for the sample appeared to be 
more heterogeneous than for the general population as evidenced 
by the fact that the standard deviation for the SBIS was 24.81 in- 
stead of the familiar value of 16.00 found on the national standard- 
ization. 

The composition of the sample from the standpoint of sex, age, 
and ability level is described in Table 1. For each sex group as well 
as for the total sample, descriptive statistics regarding chronologi- 
cal age, IQ scores on the SBIS, scores on the PM, and intercorrela- 
tions of the total scores on the SBIS and PM are furnished. 


TABLE 1 


Descriptive Statistics Regarding Chronological Age, IQ Scores on the Revised Stanford-Binet 


Intelligence Scale (SBIS), Form L-M, Scores on Raven’s Progressive Matrices (PM), 
and Intercorrelations of the Two Tests for the Two Samples of 
Men and Women and for the Total Sample 


Raven’s PM Scores 


Chronological —— Correla- 
Sample N Age Data SBIS IQ Raw Scores Centile tions of 
—— —— ——— for Two Scores 
M Range M o M* o Mean r 
63 35.1 16-49 110.2 23.2 87.5 12.2 42.5 .67* 
80 33.2 17-49 103.7 23.7 35.6 12.3 37.5 .62* 
143 34.1 16-49 106.0 24.8 36.0 12.3 40.0 .65* 


* Significant beyond .001 level. 
a The means constitute interpolated values from percentile data available in the test manual, 


Findings 

From the coefficients of correlation between the SBIS and the 
PM, which for the male, female, in total samples are seen in 
Table 1 to be .67, .62, .65, respectively, it is apparent that the two 
scales probably contain a considerable amount of common-factor 
variance. Further support for such a conclusion is also evident 
from the summary of item analysis data in Table 2, in which the 
frequency distributions of the biserial coefficients of correlation for 
the SBIS items against total scores on the PM and for the PM 
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items against total scores on the SBIS are reported. (Although not 
presented because of space limitations, the difficulty level of each 
item in the two scales as well as the coefficient of biserial correla- 
tion of each item against total score on the scale of which it is not 
a contributor are available.)? It is evident that the degree of asso- 


TABLE 2 


: 
Frequency Distributions of Biserial Coefficients of Correlation Obtained in Item Analyses : 
: 


SBIS Items with PM Items with 
Total Scores on PM Total Scores on SBIS | 
i Total Total — 
Size of Men Women Sample Men Women Sample l 
Biserial (N =63) (N = 80) (N = 143) (N =63) (N =90) (N = 143) 
Coefficients f j j f f iq 
-80 or higher 1 1 
“15.79 1 1 2 2 
-10-,74 7 0 1 2 8 3 
-65-.69 12 6 9 4 2 1 
60-64 11 15 16 4 1 3 
.55-.59 12 19 17 7 4 11 
.50-.54 13 6 11 5 13 8 
45-49 3 6 3 8 4 4 
-A0-,44 0 5 ri 8 8 11 
.35-.89 4 2 2 2 5 7 
.30-.34 3 3 2 3 3 3 
«25-29 1 1 4 r 2 
.20-.24 1 i 5 0 
.15-.19 0 0 3* 1 
-10-.14 0 1* 0 1" 
.05-.09 0 à o 3» 
.00-.04 3* 9* 5* 
68 68 68 60 60 60 


* Small size of the coefficients is 


approximately 1.00 or that the Proportion was 0.00 or close to 0.00. 


pee ea Oe 
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attributable to the fact that the Proportion passing the item was 1.00 or 7 
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ciation of items in one intelligence scale with the total scores earned 
by the examinees in the other scale was relatively high for each of 
the three groups studied. That a small number of items showed 
close to zero correlation with total score in the scale of which these 
items were not members was probably a statistical artifact arising 
from proportions either of unity (or near unity) or of zero (or 
nearly zero) answering the item correctly. Thus for very easy or 
for very difficult items the coefficients were markedly attenuated. 

When a search was made for patterns or combinations of items 
in one scale that registered high correlation with the total scores 
of the other scale, it was difficult to determine any clear-cut clusters 
of items that were associated with either a given range of correla- 
tion coefficients or with sex membership. In a detailed study of com- 
plete item-analyses which are available elsewhere (see footnote 2) 
there was a slight suggestion from the magnitudes of the biserial 
coefficients for certain items on the SBIS with total scores on the 
PM that the males tended to use verbally oriented abstract reason- 
ing in solving items on the PM, whereas the women seemed to 
rely somewhat less on verbalization and somewhat more on symbolic 
induction than did the men. Examination of the coefficients of PM 
items with total scores on the SBIS revealed no consistent pattern 
of magnitude from which any clear-cut implications could be 
drawn. 

Clearly, one of the most promising avenues for formulation of 
inferences regarding the psychological processes underlying responses 
to the PM items would be a factor analysis of the intercorrelations 
of PM and SBIS items or of short subtests of items taken from 
these scales—subtests of sufficient reliability that could be arranged 
or contrived to test certain hypotheses of cognitive function. Since 
a detailed examination of the 25 highest intercorrelations (phi co- 
efficients) of items in the SBIS with items in the PM also did not 
reveal any meaningful clusters that could be given a psychological 
interpretation, the need for factor analytic studies is indeed quite 
apparent. 


Summary 


For two adult samples of 63 males and 80 females and for the 
combined sample of 143 adults, a study of biserial coefficients of cor- 
relation of items in the Revised Stanford-Binet Intelligence Scale 
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(SBIS), Form L-M, with total scores on Raven’s Progressive Ma- 
trices (PM) scale as well as of the items in the PM with total 
scores on the SBIS revealed a relatively high degree of communality 
for the two instruments—a fact which was also supported by cor- 
relation coefficients of .67, .62, and .65, respectively, between total 
scores on the two instruments for the male, female, and total sam- 
ples. The failure of an examination of the item-analysis data and 
of the 25 highest intercorrelations of items from the two scales to 
show any clear-cut clusters amenable to psychological interpreta- 
tions pointed to the need for factor analyses of the intercorrelation 
of items or of subtests of items from the SBIS on PM scales. 
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The Sixth Mental Measurements Yearbook by Oscar Krisen Buros, 
Editor. New Jersey: The Gryphon Press, 1965. Pp. xxxvi + 
1714. $32.50. 

Expanded considerably in size and quality in harmony with the 
objectives and purposes of previous Yearbooks, The Sixth Mental 
Measurements Yearbook not only includes the major organiza- 
tional divisions of The Fifth Yearbook but also contains, contrary 
to the announced future policy in The Fifth Yearbook, comprehen- 
sive bibliographies about the construction, validation, and uses of 
specific tests and scales as well as both a detailed listing and re- 
views of tests in English-speaking countries outside the United 
States, In addition, a substantial and exceedingly expensive im- 
provement is the presentation of information regarding the status 
of tests listed in Tests in Print, but not otherwise considered in The 
Sixth Yearbook. Thus for the first time, all 1219 tests known to be 
in print as of mid-1964 are cited; 795 critical test reviews by 396 
reviewers are included along with 97 excerpts from reviews of tests 
that initially appeared in 30 journals; and 8001 references for spe- 
cific tests are incorporated. 

Information regarding the expansion of The Sixth Yearbook rel- 
ative to The Fifth Yearbook may be noted as follows: (1) the 
number of tests cited, 1219, constitutes 27.4 per cent more entries; 
(2) the number of pages is 32.6 per cent greater; (3) 628 new tests 
are incorporated; and (4) an increase of 8.6 per cent exists in the 
number of books listed in the section “Books and Reviews.” Along 
with a modest rise in price the total number of pages has jumped 
from 1292 in The Fifth Yearbook to 1714 in The Sixth Yearbook 
(excluding 36 introductory pages). A 

Another interesting set of statistics is that of the frequency of 
tests by major classification. The breakdown is shown on the 
following page. 

Despite the small percentage of multi-aptitude and achievement 
batteries in the total number of tests reviewed, the annual volume 
of such tests administered throughout the United States may well 
exceed that found for any of the other categories. Because of this 
comprehensive use Buros assigned a great deal of space to the re- 
view coverage of these two types of batteries. 
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Classification Number Percentage 
Personality 196 16.1 
Vocations 179 14.7 
Intelligence 131 10.7 
Miscellaneous 108 8.9 
English 99 8.1 
Mathematics 96 7.9 
Reading 87 7.1 
Foreign Languages 77 6.3 
Science 74 6.1 
Social Studies 61 5.0 
Business Education 29 2.4 
Achievement Batteries 28 2.3 
Sensory-Motor 27 2.2 
Multi-Aptitude 16 1.3 
Fine Arts 11 0.9 
Totals 1219 100.0 


One additional set of statistics is that presented by Buros rela- 
tive to the percentages of tests that are new or revised. As men- 
tioned previously 628 tests were new—a number which exceeded the 
445 (or 36.5 per cent) revised since the manuscript stage of The 
Fifth Yearbook. Thus the total number of new, revised, or sup- 
plemented tests was 1073 (or 88.0 per cent). 

In the spirit of previous Yearbooks the test reviews are for the 
most part critical but constructive in their presentation. As one 
would reasonably expect, inadequacies in the manuals relative to 
information furnished about validity, reliability, and standardiza- 
tion are forcefully set forth again and again. In addition to the 
valuable sections on test reviews and on books and reviews, other 


tory and index; a publishers directory and index; an index of test 
titles; an index of names of test authors, reviewers, and authors of 
articles about tests; and a classified index of tests—both those re- 
viewed in this volume and those known to be in print. 


to date. A sampling of test reviews from The Sixth Yearbook sug- 
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As on previous occasions, Oscar Buros has expressed disappoint- 
ment with the impact which the Yearbooks have had upon the de- 
velopment of discriminating taste on the part of consumers of tests. 
The message which T'he Sixth Yearbook carries apparently must be 
conveyed and supplemented by people who are in a position to com- 
municate knowledge about basic standards for educational and 
psychological tests to their consumers or probable consumers. Com- 
petent individuals in the public schools who hold positions of su- 
pervisory and administrative responsibility may be expected to be- 
come increasingly more knowledgeable about standards of 
reliability, validity, norms, and directions for administering tests— 
especially as educational training in leading colleges and universi- 
ties is improved. But perhaps the group of individuals who have the 
greatest responsibility are professors of psychology and education 
who in their classes and inservice workshops conducted off campus 
have the almost missionary obligation to reach as many potential 
consumers of tests as they possibly can. Once students have gained 
the necessary basic knowledge about the requirements of good tests 
they should be directed to read and to reread the Buros Yearbooks 
prior to making any decisions regarding the adoption of standard- 
ized tests for use in schools, industry, or clinical settings. Perhaps in 
a few years, with such concerted efforts on the part of professional 
leaders in educational and psychological testing, Oscar Buros’ pro- 
jected Seventh Mental Measurements Yearbook will accomplish for 
the rank and file test user what he so ardently had hoped would be 
achieved many years ago with earlier editions of the Yearbooks. 

Joan J, BJELKE 
University of Southern California 
AND 
Wurm B. MICHAEL 
University of California, Santa Barbara 


Porteus Maze Test: Fifty Years’ Application by Stanley D. Por- 
teus. Palo Alto, California: Pacific Books, 1965. Pp. 320. 

Stanley Porteus’ life by almost any standard has been a highly 
successful one. He was vouchsafed fairly early an original insight 
in psychometrics and has carefully nursed his brain child along for 
over fifty years of application, research, and controversy. He de- 
veloped his maze test in 1913, described its first application at the 
British Association for the Advancement of Science meeting at the 
University of Melbourne in August, 1914, and published his first 
papers on the topic the following year. The inception of the Porteus 
Maze Test and World War I was coetaneous. 

In 1919 Porteus came from Australia to succeed Goddard at the 
Vineland Institute in New Jersey; then a few years later he went to 
the University of Hawaii, from which he has long retired as pro- 
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fessor emeritus. He has never been accorded adequate professional 
recognition among American psychologists, possibly because he 
was something of an Ausländer among the natives. It seems more 
probable, though, that his espousal of tests as a touchstone for 
evaluating racial differences in intelligence in the long dead contro- 
versies of a generation back had more to do with the lack of ac- 
ceptance of his mazes and the claims he has made for them. One 
should think that a racial melting pot such as Hawaii would find 
such a set of attitudes especially difficult to empathize into or ac- 
cept. Finally, the mazes ran head on into the Binet-type test which 
had only recently stirred up excitement in the United States prior to 
Porteus’ arrival at Vineland. For 20 years after its publication in 
1916 the Stanford-Binet Intelligence Scale was accepted as the 
true measure of intelligence, against which any newcomer must be 
validated. Thus it can be seen that Porteus and his mazes gave 
many hostages to fortune while making a bid for professional rec- 
ognition in the United States: Porteus was “foreign,” he chose un- 
popular causes, and Terman’s Stanford-Binet Intelligence Scale was 
already entrenched. Only through long life, energy, and persistence 
have these hostages been neutralized. This book tells the story—a 
successful one—of the tangled Odyssey of the mazes. 

The mazes began, as did the 1905 Binet scale, with a practical 
problem, the screening out of feebleminded children so that (it was 
hoped) they could more effectively be instructed in specialized 
schools. Porteus felt in 1913 that Goddard’s version of the Binet 
Scale was a very imperfect measure: the mazes were developed to 
aid in the differential diagnosis of the fecbleminded. Porteus be- 
lieved that the performance measure he invented in the maze test 
cut through the pitfalls of the purely verbal measure and got at 
kome native aptitudes ofa practical nature, especially of “planfull- 
ness.” It has taken him a lifetime to document his original hunch 
but he has done it in a number of specific ways. i 

Aside from Porteus’ own publications, one of the good early con- 
firmations of the special extra quality to be found in the mazes but 
KE i oe Binet type test was a study of Poull and Montgomery in 
ae Le oe be Paor Bian Intelligence Scale and the 
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the mazes which he published—a scoring system showing marked 
validity as a means of discriminating prison populations from more 
nearly normal groups. There has been repeated confirmation of 
Porteus’ original findings. One of the more noteworthy was pub- 
lished in 1954 by Docter and Winder from Stanford. The mazes 
would appear to measure something both quantitatively and quali- 
tatively which discriminates against the impulsive individuals so 
frequently found among the delinquent and criminal. 

The Porteus Maze Test was found by Porteus and Peters, later 
by Landis and Zubin to be a sensitive measure of the cerebral insult 
of psychosurgery. Whereas the Wechsler-Bellevue Intelligence Scale 
showed no impairment subsequent to the operation, the mazes did. 
And Porteus has subsequently made the chilling point that if the 
practice effect on his mazes is taken into account, the damage is 
permanent in most but not in all cases. It is fortunate that this 
pioneering blind alley of the neurosurgeons was so soon tossed into 
the dustbin of history through the advent of chemotherapy. 

Porteus also demonstrated from the beginning that the maze test 
is a measure much to be preferred to the traditional intelligence test 
when institutional groups—the feeble of mind, the criminal, or the 
delinquent—are to be sorted for training or treatment. It seems 
likely that the planfullness aspect, presumably calibrated by the 
mazes, is the significant parameter in its evident superiority as a 
test to the Binet and the Wechsler type of measure in these institu- 
tional areas. 

In several anthropological expeditions, some by Porteus, some by 
others, the mazes have been found to be a type of measure which 
pre-literate groups accept and find interesting. Validation of the 
mazes for such groups is, of course, fragmentary. What has been 
reported makes it seem likely that the test is a valuable adjunct for 
the anthropologist to know about and to employ for research pur- 
poses in his investigations. 

Most recently Porteus has devised a conformity-variability 
(C-V) for his mazes, a projective kind of measure which quantifies 
the degree to which the same kind of attack is used on the mazes 
when they are re-taken. Individuals can be matched successfully, 
he says, in about 90 per cent of the cases. If the conformity score is 
too high, Porteus feels that the person is constricted, meticulous, 
and compulsive; too low, and he suspects a varying amount of dis- 
organization. Here are some hypotheses by the youthful octagenar- 
ian—those in need of a problem, let them try their hand at testing 
these brain children of the Ancient of Days from Aloha Land. 

In 99 out of 105 studies Porteus has found the male to have a 
maze test mean superior to that of the female. Porteus does not 
exactly claim that this shows general male superiority of intelli- 
gence; he suggests that it probably represents inherent tempera- 
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mental differences. Since he has so often equated temperamental 
differences with manifest intelligence, one wonders whether he does 
not feel about women’s mental inferiority somewhat the way he did 
about the various non-Caucasian groups in Hawaii and their nu- 
merous crosses which he tested and found wanting a generation 
ago. One hastens to add that this claim is not made in this latest 
book of Porteus. It may be that he has succumbed to the Zeitgeist. 
It may also be that he feels it is better not to tilt at windmills with- 
out allies, 

We should be thankful to Porteus for his mazes and for his re- 
search, He is still a vigorous, undaunted sample of what originality, 
persistence, hard work, and long life can bring about. Let us hope 
tuha brings us up-to-date once more when he reaches the century 
mark. 

Wam D. ALrus 
University of California, Santa Barbara 


Statistical Inference I, by Jerome C. R. Li. Ann Arbor Michigan: 
Edwards Brothers Inc. 1964. Pp. xix + 658. 

Statistical Inference appears in two long volumes. Only Statisti- 

cal Inference I, the revised edition of Introduction to Statistical 
Inference, which was published in 1957, is reviewed here. 
_ Principal changes in the revised edition include a very welcomed 
improvement in printing and the unexplained loss of excellent sets 
of questions for most of the chapters. (Sets of exercises with solu- 
tions have been continued.) Two new chapters appear at the end of 
the new edition. A third review chapter has been added and is fol- 
lowed by a chapter which discusses outlying observations, homo- 
geneity of variances, and determination of sample size. 

_ The book is a non-mathematical exposition of the theory of sta- 
tistics written for experimental scientists, Many of the examples in 
m text end in the ye are drawn from psychology and edu- 
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Two review chapters divide the book into three parts. The first 
part (Chapters 1-13) of the book presents the basic concepts of 
statistical inference and introduces the normal, x°, t, and F distri- 
butions. Sections are devoted to the “u-tests” (‘z-tests”), the “t- 
test” of paired observations, the “t-test” of independent groups, the 
x° test of population variance, the “F-test” of two population vari- 
ances, one-way analysis of variance, and confidence intervals. 

This review is written after the reviewer’s having used Li’s book 
in several classes of intermediate statistics taken primarily by psy- 
chology majors. Students have been almost unanimous in their 
praise of the clarity of presentation of the material in part one. This 
remarkable state of affairs appears to be the result of the great care 
that Li has taken with sentence structure and transition and his 
decision to exclude many supporting and tangential topics in order 
to focus attention on the concepts at hand. One does not find any 
general introduction to probability, permutations and combinations, 
the rules of summation notation, and the topic of expected values. 
However, for the student who has just stumbled through a mathe- 
matically elegant introductory course in statistics, this book offers 
an opportunity to discover what statistics is designed to accom- 
plish. The instructor in an introductory class will probably want to 
supplement and provide additional background material at some 
points. 

One perplexing change of level of presentation is found in part 
one. In a well written chapter, the x? distribution (chapter 7) is 
presented as a general model which is generated by repeatedly 
summing k independent values of u?, where u is defined as a stand- 
ardized normal deviate. The statistic SS/o? is then mathematically 
shown to follow this general model. Once the necessary background 
has been presented, one would then expect that the t and F distribu- 
tions would also be defined in similar terms and that the statistics 
that bear their names would be shown to conform to the general 
model. Instead, Li returns to the use of sampling experiments to 
convince the reader that the statistics have been properly named. 

The second part (Chapters 14-20) presents the topics of random- 
ized blocks, tests of specific hypotheses in the analysis of variance, 
linear regression, factorial experiments, and analysis of covariance. 
Repeated measurements designs are neither presented nor men- 
tioned either with regard to three or more treatment conditions in 
this part of the book or in connection with the “test-test” of paired 
observations in part one. 

The presentation of each topic includes a limited discussion of 
experimental design. This is a common practice in many useful 
statistics texts. However, a general discussion of experimental de- 
signs, perhaps as an overview before specific methods of analysis are 
presented, would help the student to develop a framework for the 
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methods of analysis. This would be particularly important for 
commonly used designs that otherwise are not covered. 

Sampling experiments are used extensively to verify the sam- 
pling distributions of the statistics that are considered. The im- 
portance of an individual degree of freedom is stressed and helps to 
provide continuity from topic to topic. A basic part of Li’s approach 
is to relate the separate topics. This is accomplished with con- 
siderable skill in two valuable sections, one which compares and 
contrasts linear regression and analysis of variance, and another 
section which discusses the relationships between analysis of co- 
variance and factorial experiments. 

The chapter on factorial experiments quickly represents the two 
dimensional case and discusses the differences between the fixed, 
mixed, and random models. An important error of omission occurs 
in that no discussion is offered on how to proceed in the fixed model 
case when the interaction is significant. 

Part three (chapters 21-25) is primarily concerned with extend- 
ing, through the vehicle of analysis of variance, most of the meth- 
ods of analysis that were developed in the first two parts to 
sampling from one or more binomial populations. The chi-square 
test of goodness of fit for one or more samples from binomial popu- 
lations and the chi-square test of independence for two binomial 
samples are shown to be directly related to analysis of variance 
with appropriate modifications and restrictions. Sampling experi- 
ments are used to show that the statistics that are computed from 
the chi-square test of independence for two or more multinomial 
populations and the chi-square test of goodness of fit for one mul- 
tinomial populations approximately follow the x’ distribution. By 
using this approach, Li gives these topics comparatively thorough 
treatment in relatively short space and provides a link between 
sampling from normal and from binomial populations. The reader 
who has digested the topics of analysis of variance, linear regres- 
sion, and the use of a single degree of freedom, can appreciate the 
degree of unity that Li has brought to the parametric and non- 
Parametric methods that are discussed. 
fy aT ra book is a solid and eminently readable presen- 
t i y pics that are often presented in introductory and 
intermediate courses in statistics. On the strength of the clarity or 
the exposition alone, the book can serve many functions. The in- 
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on multiple regression and its ramifications. Finally, for the in- 
structor who will probably never be completely satisfied with any 
one statistical text, Li’s book can serve very satisfactorily in giving 
support to and in freeing classroom time for individual approaches 
and emphases in statistics. 

EMANUEL LASK 

California State College at Los Angeles 


Exercises to Accompany Fundamental Statistics in Psychology and 
Education (4th Edition) by J. P. Guilford, William B. Michael, 
and Stephen W. Brown. New York: McGraw-Hill, 1965. Pp. 
vii + 204. $3.50. 

This set of exercises and problems is essentially a collation of 
those from the third editions of the authors’ Elementary Statisti- 
cal Exercises and Intermediate Statistical Exercises. Some appro- 
priate new material has been added with the intention of providing 
a single comprehensive accompaniment to the fourth edition of 
Guilford’s Fundamental Statistics in Psychology and Education. 

The aim of a good workbook should be to include as many rele- 
vant exercises and problems as possible, not to provide large num- 
bers of beautifully laid out but uninformative worksheets. In this, 
Guilford, Michael, and Brown have succeeded very well. Instruc- 
tions and directions for problem lay-out are clear but often im- 
plicit. The accounting approach to statistical computations is 
largely deemphasized in favor of a “what’s going on?” approach. 
The exceptions to this are the inordinate attention paid to coding 
of observations and to the trial balance method of computing the 
mean and standard deviation, and the inclusion of large and 
clumsy (even if traditional) tables for calculating a product- 
moment correlation coefficient. With the availability of modern 
computing techniques and devices, these emphases cannot be justi- 
fied. The section on analysis of variance is much less heavily 
weighted than the current interest and importance of this topic 
demand in a general introduction. However, the faults mentioned 
are more reflections of weaknesses in Guilford’s fourth edition than 
of the workbook itself. Taken alone, Exercises to Accompany is an 
excellent workbook which should provide students very adequate 
operational experience in statistical procedures and statistical 
thinking. 

James A. WALSH 
Iowa State University 


Measurement and Evaluation in Teaching by Norman E. Gron- 
lund. New York: The Macmillan Company, 1965. Pp. xii + 
420. $6.95. 

Gronlund’s purpose is to introduce the reader 
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. . . to the principles and procedures of evaluation which 

are essential to good teaching. The main theme which runs 

throughout the book is that evaluation is an integral part 

of the teaching-learning process and that it involves three 

fundamental steps: (1) identifying and defining instruc- 

tional objectives in behavioral terms, (2) constructing or 

selecting evaluation instruments which most effectively ap- 

praise these specific learning outcomes, and (3) using the 

results to improve learning (p. vii). 
Centering his book upon the place of evaluation in teaching, rather 
than upon principles of making and interpreting tests as though 
they were ends in themselves, Gronlund avoids the shortcoming of 
writers of introductory texts who leave to the immature student 
the task of integrating the concepts of the psychology of learning, 
methods of teaching, and tests and measurements. Gronlund is 
second to none in achieving this needed synthesis and succeeds in 
portraying evaluation as an integral part of teaching. 

Emphasis upon computational work is avoided. Abilities in inter- 
preting and applying statistical concepts that are essential to effec- 
tive use of tests are the outcomes sought. Rank-order correlations 
and expectancy tables are introduced logically in connection with 
the discussion of validity. The normal curve and standard devia- 
tions are capably presented within the context of standard score 
norms; however, “since the method of computing the standard de- 
Viation is not especially helpful in understanding it, the procedure 
will not be presented here” (p. 286). 

No attention is given to the history of measurement. The names 
of such pioneers as Binet, Terman, and E. L. Thorndike do not 
appear. While historical content is often interesting (to instructors, 
at least) , its omission from a text which is properly concerned with 
improving the ability of teachers to evaluate their pupils seems 
well advised. 

Part I deals with the evaluation process. Superb examples are 


given of developing behaviorall stated objecti i f 
subject fields that t y objectives for a variety o: 


purpose when viewed as a means 
tives rather than as an end in itself” 
develops skills in defining educational 
haviorally. This chapter, which is prob: 
the skills of writing objectives, 
riticiz behavioral objectives. Well-chosen illustra- 
ds x ne 3 relate specific evaluation items to behaviorally 
k a : jectives, The significant influence of evaluation upon 
udent learning is stressed. The discussion of content, predictive, 


is related to the instructional functions of 
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classroom teachers. Construct validity is handled so skillfully as to 
minimize confusion for the uninitiated. 

A conventional presentation of strengths, limitations, and sug- 
gestions for writing each kind of item is given in Part II. Well- 
chosen examples of good and poor items illustrate common pitfalls 
of item writing. Chapter bibliographies provide excellent sources of 
sample test-items for various elementary and secondary subject 
fields. Chapter 9 skillfully develops the use of the interpretative 
exercise. Text and illustrations combine to elucidate the measure- 
ment of such abilities as recognizing the relevance of information; 
applying principles; recognizing warranted and unwarranted gen- 
eralizations, assumptions, and inferences; interpreting experimen- 
tal findings; and utilizing graphs, cartoons, and maps. This empha- 
sis upon objective means of assessing highly transferable learning 
is much needed. 

Part III deals with the use of standardized tests. Readers are 
spared the lengthy descriptions of instruments that often clutter 
introductory measurement texts. Students are provided with means 
of finding, evaluating, selecting, and using the instruments they will 
need. Transferable skills, rather than specific knowledge, are em- 
phasized. 

Two faults appear to mar Gronlund’s treatment of Chapter 14, 
Interpreting Test Scores and Norms. In this well-written but stand- 
ard explanation of various ways of reporting scores and using ref- 
erence groups, Gronlund fails to make a clear distinction between 
the meaning of the word “norm” as it pertains to reference groups 
and the same word as it pertains to ways of reporting scores—a 
not uncommon failing. It is the reviewer’s unhappy experience that 
neophites often confuse the two meanings unless the difference is 
explicitly clarified. The other exception taken with this chapter 
refers to the statement that “A distinctive feature of percentile 
norms is that we can interpret a pupil’s performance in terms of 
any group in which he is a member, or desires to become a member” 
(p. 284). A splendid discussion of the merits of using multiple ref- 
erence groups follows. One might challenge the assertion that the 
merits and use of multiple norm groups are either necessarily or 
logically limited to percentile norms. 

In Part IV, Evaluating Procedures, Products, and Typical Be- 
havior, clinical tools are omitted from the discussion, in favor of in- 
struments which typical teachers might use. For instance, the 
MMPI is not mentioned, and projective techniques are summa- 
rized in three paragraphs. 

Part V consists of only one chapter. The first part discusses the 
role of evaluation in improving learning; the second part presents 
practical and useful suggestions for improving marking and report- 
ing. 
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In summary, Gronlund has given us an extremely well-written 
textbook in educational evaluation, the principal theme of which is 
highly appropriate for introductory courses in educational evalua- 
tion and tests and measurements. It achieves its purpose of intro- 
ducing teachers to the principles and procedures of evaluation 
which are essential to good teaching. 

Grratp S. Hanna 
University of Alaska 


Psychological Test Modifications by Milton Kornrich (Editor). 
Springfield, Illinois: Charles ©. Thomas, 1965. Pp. xii + 265. 
$8.75. 

Users of projective techniques are frequently faced with the 
problem of deriving clinical hypotheses from sparse and guarded 
test protocols. One alternative to this problem in clinical psychol- 
ogy is to devise more objective approaches to personality assess- 
ment. This volume, however, takes the point of view that although 
there is untapped mineral wealth in the testing situation, the usual 
mining tools require some modifications. Dr. Kornrich has brought 
together eighteen articles, three previously unpublished, which de- 
scribe modifications in the test administration of the standard pro- 
jective techniques to augment the results of the original procedure. 

As might be anticipated, most of the articles (seven in all) deal 
with modifications of the Rorschach. Three articles are devoted to 
the Thematic Apperception Test (TAT) as well as to measures of 
word association, two to the Bender-Gestalt, and one apiece to the 
Children’s Apperception Test and drawings. Several of the newer 
procedures call for eliciting associations to responses obtained in 
the standard testing session. Perhaps the most unusual method is 
that of Richard M. Jones’ “negated response” technique for use 
with the TAT. Other procedures involve joint husband and wife 
testing with the Rorschach and self-interpretation of the TAT. 

Several techniques appear to be particularly interesting and per- 
haps promising. Jones’ negated responses and Howard M. Hal- 
pern’s Rorschach interview technique stand out as procedures that 
would seem to merit further investigation by clinical psychologists. 
The usual methodological difficulties in validating projective tech- 
niques, however, once again confront the reader, 

It would be a simple task to attack the book for being weak in an 
area not intended to be strong by the editor in the first place. This 
can be done by pointing out the glaring lack of consideration for 
i validities and reliabilities of the techniques presented. With the 
exception of one paper by Jones, supporting studies are missing. 
However, one has to consider the spirit in which the collection is 
offered. The “dearth of current and past research” is acknowledged 
in the editor’s preface. The purpose is to bring to light some pro- 
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cedures which may prove helpful in the clinical situation and which 
merit further study. 

Although Dr. Kornrich’s collection may offer little to the psy- 
chologist concerned with test theory, clinical psychologists may 
find the book to be lively and provocative reading, particularly if 
viewed as intriguing suggestions rather than as a clinical manual. 

PHILIP HIMELSTEIN 
Texas Western College 


Widening Horizons in Creativity (The Proceedings of the Fifth 
Utah Creativity Research Conference) by Calvin Taylor (Edi- 
tor). New York: John Wiley and Sons, 1964. Pp. xix + 466. 

Furnishing a report of the Fifth (1962) Utah Creativity Re- 
search Conference held at the secluded and scenic locale of the 
Mount Majestic Lodge and Manor in the Wasatch Mountains, 
Widening Horizons in Creativity consists of five parts and a total 
of twenty-nine chapters by a group of distinguished research spe- 
cialists and writers in the area of creativity. The published papers 
represent a broad sampling of research contributions from many 
points of view. Surprisingly enough, even without an overview 
chapter, there is a marked degree of unity and coherence in the 
volume that probably reflects the painstaking care exhibited by the 
editor in his organization of the proceedings. 

Titled “Historical Reports,” Part I contains two reprinted papers 
by Toynbee and by Thurstone who were not participants at the 
Conference. Although written nearly nine years before the Confer- 
ence was scheduled, Thurstone’s paper constituted an amazingly 
accurate forecast of the problem areas, research methodologies, and 
substantive contributions to be encountered in later investigations 
on creative scientific talent. Thurstone’s hypotheses were highly in- 
sightful forerunners of what is being subjected to systematic in- 
quiry today. 

Identified as “Creative Process Studies,” Part TI consists of seven 
papers that range from Ghiselin, Rompel, and Taylor’s check list of 
activities associated with the creative process to the clinically 
oriented paper by Barron on the relationship of ego diffusion to 
creative perception, to say nothing of Leary’s contribution regard- 
ing the influence of test score feedback on creative performance and 
of drugs on creative experience. In addition, there are papers by 
Westcott on empirical studies about intuition, by the Mednicks on 
an associative interpretation of the creative process, and by Hyman 
on the role of information and induced attitudes in creativity. The 
concluding paper in Part II is a free-wheeling, spontaneous discus- 
sion by the Conference participants on process versus product in 
creativity. 

Of considerable interest and value to the professional educator 
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in the college or university as well as to the public school teacher 
with adequate psychological background, Part III, “Education and 
Development of Creativity” is made up of six papers. Torrance 
reviewed the significant contributions of the Minnesota studies to 
creative thinking, and Parnes summarized several evaluation stud- 
ies on research carried out in school settings to develop creative 
behavior. In his clinically oriented investigation of samples of 
psychologists whom he designated as creative, noncreative produc- 
tive, noncreative nonproductive, and combined control groups, 
Drevdahl noted substantial differences in prior home and educa- 
tional environments as well as in the current patterns of work 
habits of these groups. Although not directly concerned with crea- 
tivity, Harmon’s paper on career determiners of high-level per- 
sonnel yielded validity information on five predictors against a rat- 
ing of scientific accomplishment. How to develop creative research 
performance in public school children was the subject of Jablinski’s 
largely anecdotal and impressionistic report. In a related paper 
Brust discussed the development of readiness in students in the 
primary and elementary grades for subsequent creative explora- 
tions. 

Perhaps of greatest relevance to the orientation of the majority 
of readers of EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT Was 
Part IV, “Criterion and Prediction Studies.” Taylor and Ellison 
discussed the prediction of creative performances from use of mul- 
tiple measures; Guilford expounded on the progress of his syste- 
matio research program involving the discovery of intellectual 
factors; Holland reviewed his work on the assessment and predic- 
tion of creative performance of high-aptitude youth; Mullins sum- 
marized the findings of four predictive studies involving use of 
various types of creativity measures; Sprecher described the extent 
of individual differences in numerous criterion measures for engi- 
neers who were being judged for their creative performance; and 
Astin attempted to conceptualize types of variables for creativity 
research in terms of high versus low social relevance and natural- 
istic versus manufactured properties. 

For the lay reader as well as for the lay educator, the eight pa- 
pers in Part V, “Creativity in Special Fields and Settings,” prob- 
ably hold the greatest stimulus value. MacKinnon reviewed his 
familiar work on creativity in architects; Beittel insightfully con- 
sidered creativity in the visual arts at the college and university 
level by showing an amazing grasp of the interrelationship of psy- 
chological knowledge and artistic performance; Elliot briefly de- 
scribed the measurement of creative abilities in public relations and 
advertising work; Fielder outlined a number of studies concerned 
with the effect of group climate on creative group performance; 
Gamble summarized the kinds of interest shown and efforts ex- 
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pended by the National Aeronautics and Space Administration 
(NASA) in selecting scientists and engineers; McPherson pointed 
out prospects for future creativity research in industry; Lois-ellin 
Datta reported on the observations of a committee from industry 
on the Creativity Conference; and Levine expressed what the ef- 
forts and interests were of the National Science Foundation in the 
development and utilization of creative talent. 

Perhaps of as much interest as the main contributions of each of 
the chapters are the extensive discussions and interchanges to be 
found within several of the chapters between the speaker (author 
or co-author of a chapter) and participants at the Conference. 
These give-and-take sections are not only helpful in clarifying 
many of the points made by the author or authors of a given chap- 
ter but also fertile in suggesting numerous hypotheses about the 
nature of and the approaches to the study of creativity. Another 
source of valuable information to the reader is the bibliography of 
approximately 300 citations. 

For the professionally trained psychologist as well as for many 
a lay person this carefully prepared and edited volume affords a 
broad overview of the disparate but insightful ideas concerning 
the processes and products of creative endeavor. As a survey of 
recent developments in the study of creativity this book probably 
furnishes one of the most comprehensive and authoritative treat- 
ments currently available, After reading this significant book, one 
can hope that within another four or five years the proceedings of 
future Utah conferences will be as adequately described as have 
been those of the Fifth Utah Creativity Research Conference. 
Hopefully, if a new volume is forthcoming in 1970, its audience 
will have a basis of assessing and evaluating changes in research 
efforts on creativity since 1962 and for determining the probable 


directions that future research efforts may be expected to follow. 
Wim B. MICHAEL 


University of California 
Banta Barbara 


Modes of Thinking in Young Children: A Study of the Creativity- 
Intelligence Distinction by Michael A. Wallach and Nathan 
Kogan. New York: Holt, Rinehart and Winston, Ine., 1965. Pp. 
viii -+ 357. 

This volume contains an investigation categorized in a general 
sense as systematic observation of the psychological ability called 
creativity. The authors, Wallach, a member of the Psychology De- 
partment at Duke University and Editor of the Journal of Per- 
sonality since 1963 and, Kogan, a Senior Research Psychologist 
with the Educational Testing Service, have mainly questioned: 
Can solid evidence be found that will support the validity of a dis- 
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tinction between the traditional concept of general intelligence and 
“creativity” as modes of cognitive activity? i f 

Throughout examination of this important research it is essential 
to keep the operational definition of creativity used in this study 
well in mind. Mainly the dimension is an associational one, asso- 
ciative flow, the production of associative content that is abundant 
and that is unique. A further requirement was that the children in 
the study should generate unique and plentiful associates in a gen- 
erally task-appropriate context which is permissive and playful. To 
call a response unique for a given item is to say that the response 
in question was provided in answer to the item by only one child in 
the entire sample of 151 fifth-grade pupils. For example, when re- 
quested to tell all the different ways to use a newspaper, “rip it up 
if angry” was a unique response, while “make paper hats” was not 
(p. 32). For an inverted “v” shaped line in the Line Meanings in- 
strument, “private’s stripe” was a unique response, but “hat” was 
not (p. 23). 

Concerning the question of whether a response that is unique in 
the sample as a whole can be considered a creative product for the 
child, the authors propose that there is a substantial degree of cor- 
respondence between “actuarial” and “personal” uniqueness when 
the sample derives from a reasonably homogeneous sociocultural 
matrix, such as was the case for the present sample. With regard to 
the hypothesized concomitance of uniqueness and fluency, in the 
case of four of the five “creativity” procedures, scored for unique- 
ness of responses and scored for number of responses, the results 
were significantly related. Further, the authors expected stereo- 
typed associates to come earlier and unique associates to come later 
in a sequence of responses. In that case, an appropriate assessment 
context required freedom from pressure of short time limits, even 
freedom from any temporal pressure at all, and no evaluative pres- 
sure. Consequently children could focus upon the task rather than 
upon the self. Towards the end of the last chapter, the authors ad- 
mit that the enumeration of alternative possibilities is but an in- 
termediate step—a precondition for the creative act, and that they 
do not imply that such a precondition for creativity in a child will 
necessarily lead to actual creative performance, such as creative 
problem solving. ; 

The procedures of the study may be summarized briefly as fol- 
lows. In order to examine the creativity-intelligence distinction and 
the elucidation of possible psychological correlates of individual 
differences, Seventeen different measures or procedures were ad- 
ministered to the sample of six classes, 70 boys and 81 girls, from 
o New England schools. These measures (three verbal, two vis- 
Ua aenea scare fom, cognitive syle Gand wii), eno. 

Physiognomie properties of stimuli), relatively 
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enduring motivational sets of defensiveness and anxiety, and in- 
telligence (ten indicators). The measures were reported to be highly 
reliable—in terms of split-half and item-sum correlations. In fu- 
ture study it would be desirable to know test-retest reliabilities for 
the “creativity” measures. Besides the seventeen measures, observa- 
tions were made for a two week period to gather behavioral data on 
the children and a clinical description was made of thirty-two of 
the children. 

Upon finding that the “creativity” measures were highly inter- 

correlated (on the order of .4) (an achievement that the authors 
failed to find in reviewing related research) and upon finding that 
the correlations between the “creativity” and intelligence measures 
proved to be extremely low (on the order of .1), the authors then 
composed four groups of children within each sex: those high in 
both “creativity” and intelligence, those high in one and low in the 
other, and those low in both. The number of subjects in each of 
these cells was approximately 17 or 18. A single “creativity” and 
“intelligence” score was obtained for each child by summing the 
standard scores of the measures in the respective domains—possi- 
bly a questionable practice in spite of .4 intercorrelations. To yield 
the groups with the four possible combinations of “creativity” and 
intelligence, the respective scores were dichotomized at their me- 
dians, Next, analysis of variance procedures were used with the 
other psychological measures to ascertain distinguishing differ- 
ences. 
The following is a sampling of a few of the many findings. Be- 
sides the finding of the isolation of an associational dimension of 
creativity independent of conventionally defined general intelli- 
gence, which the authors felt was largely fostered by the permis- 
sive and playful test situation, the child’s overt behavior in the 
classroom appeared to vary as a function of his “creativity” and 
intelligence. For example, although an advantage was found for 
those who were high in both “creativity” and intelligence, this 
group was high also in regard to disruptive, attention-seeking be- 
havior (p. 194). In many respects the group high in creativity but 
low in intelligence was at the greatest disadvantage. : 

Related to conceptualizing activities, the findings, which may be 
influenced by ipsative properties of the measurement, focused on 
the importance of thematizing abilities for “creativity.” “Creative” 
boys seemed able to switch rather flexibly between thematizing and 
inferential-conceptual bases for grouping (distinctions derived from 
work by Kagan, Moss, and Sigel). The authors’ review of the litera- 
ture appeared to omit some important studies of the abstract, func- 
tional, concrete dimensions, such as, the research by Russell and 
Saadeh and by Reichard, Schneider, and Rapaport, to mention a 
few, stemming from developmental theory of Piaget. After their 
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(possibly inadequate) survey of the literature, the authors re- 
jected the importance of the abstract dimension of cognitive style 
as related to intelligence and as possibly related to creativity. È 

In regard to the question of how children set limits or boundaries, 
the major finding was a relationship between “creativity” and the 
willingness to tolerate deviant instances as possibly warranting 
membership in the category, i.e., wider tolerance limits. 

To turn to some evidence on how the children describe them- 
selves with respect to anxiety, anxiety level was highest for the low 
“creativity-low intelligence group. If anxiety was either too low or 
too high, then “creativity” was reduced. On the other hand, the 
kind of disturbance represented by defensiveness was not conducive 
to “creativity” in boys. 

Lastly, the ability to produce unique and plentiful ideational asso- 
ciates showed commonality with capacity to apprehend expressive 
characteristics potentially carried by visual stimuli (physiognomic 
properties). The case studies tended to support the quantitative 
results. Explanations for poor performance in one thinking mode, 
when performance in the other is superior, centered upon motiva- 
tional interferences, 

Perhaps the major contribution of this volume stems from the 
attempt to design research so that depth of investigation is suc- 
cessfully combined with reasonable breadth, so that both academic 
psychology and educational research are brought into closer rela- 
tion, and so that the psychological study of cognitive processes is 
brought into greater contact with education, At the same time, ex- 
treme methodological care and rigor are exhibited in this study. 
The authors should be commended for attempting to relate their 
findings to possible school practice (last chapter). Their emphasis, 
however, is possibly more appropriate for basic or high level di- 
mensions of creative thinking. Perhaps the most significant con- 
tribution of this work is that it paves the way for more complex, 
more relevant aspects of creative thinking to be examined. The 
authors might have grasped this opportunity themselves with one 
of their Measures originally designed to evaluate defensiveness—a 
series of unfinished problem-stories, which could have been used to 
stimulate creative problem-solving behavior in the children. This 
avenue should be pursued further, Instead, in the reviewer's opin- 
ion, most of the “creativity” procedures used in this study smacked 
of the laboratory game with trivial tasks having little relevance to 
the life of children, It is hoped, in addition, that future studies will 
emphasize and build upon knowledge of larger theoretical schemes 
of children’s cognitive development. 

The implications from this study 
cal measurement may stem largely 
phere. It is wondered whether the 


for educational and psychologi- 
from the unique testing atmos- 
distinction would be so sharply 
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maintained if the intelligence measures could have been given in 
the same playful, permissive, untimed, supportive, atmosphere as 
the creativity tests were given. It is wondered what effect, if any, 
results from creativity measures being given in abundance and be- 
fore intelligence tests. 

This study should not be underestimated and should be assimi- 
lated and accommodated into present understandings from earlier 
research on children’s creative thinking. In earlier studies claims, 
too frequently, have exceeded achievements. The findings of this 
_ study indicate that “creativity” denotes cognitive functioning that 
is of importance in the search for new knowledge. The authors have 
supported the point that to ignore the challenge of measuring par- 
ticular children with regard to creativity interacting with intelli- 
gence is to ignore much of the cognitive potential of young chil- 
dren. 

Sara W. LUNDSTEEN 
University of California, Santa Barbara 


Business Data Processing by Elias M. Awad. Englewood Cliffs, 
New Jersey: Prentice-Hall, Inc., 1965. Pp. x + 310. $7.55. 

Awad’s book, the preface indicates, “is designed for a basic course 
in data processing, for first- or second-year students in schools or 
colleges of Business Administration. . . . The objective of the book 
is to provide the student of business administration with a basic 
and thorough understanding of data-processing principles. .. . In- 
cluded are principles related to both punched-card and electronic 
data processing equipment.” 

In general this text fits the group toward which it is aimed and 
accomplishes its purpose. This reviewer is particularly impressed by 
the thorough (perhaps too thorough) treatment of punched-card 
equipment. Awad has avoided the weakness of many introductory 
data processing texts: a perfunctory reference to punched-card 
machines and an overemphasis on the more glamorous computers. 
He strikes a more suitable balance for a basic, general text—de- 
voting 8 out of 22 chapters to the older “traditional” equipment, 
and 9 to electronic computers. In rather typical but inescapably 
logical fashion, Chapters 1 and 2 establish the need for efficient 
data processing techniques and trace the history of development of 
computing machinery. In Chapter 1 Awad makes a better than 
average distinction between punched-card and computer methods 
of data processing, and presents a better than average explanation 
of the distinction between digital and analog computers. The his- 
torical development of Chapter 2 is interesting and complete. 

In Chapters 3 through 10 each of the basic punched-card ma- 
chines is described and its function explained. Inevitably the reader 
is thrust into some of the complexities and technicalities of the 
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workings of the equipment. This section is well illustrated, however, 
(as is the entire book), and it should provide a good understanding 
of the applications of punched-card equipment to business data- 
processing problems. 

Chapters 11 through 20 present the basic capabilities which a 
machine must have to be a computer as well as discuss in somewhat 
excessive and at times obscure detail some of the mechanical and 
electronic features of computers. The explanation of the process of 
reading and writing in magnetic core memory units (pp. 167-168), 
admittedly technical and difficult to explain, would be unintelligible 
to the majority of readers. It would be better to omit this “explana- 
tion” if it cannot be presented more clearly. 

The section on coded decimal schemes (Chapter 17) is of doubt- 
ful value, although the topic is almost universally discussed some- 
where in other introductory data processing texts. The beginning 
specialist or general interest reader has no need for this knowledge. 
In the experience of this reviewer he is usually confused by it. 

Many first level data processing texts devote considerable space 
to the principles of computer programming, sometimes based on a 
hypothetical machine of the author’s, or sometimes using the in- 
struction format (and the inevitable accompanying peculiarities) of 
an actual computer. Data processing teachers and textbook authors 
alike are uncertain as to the desirability of including this material 
in beginning texts. Awad handles the dilemma by briefly presenting 
(in Chapter 19) a simple example of how programming is done on 
the IBM 1401, a widely used business computer. Individual in- 
structors could elaborate on this with a more intensive treatment 
of programming, as could the casual reader, if so inclined, by aug- 
menting this chapter with 1401 programming manuals available 
ie IBM or by reference to a number of texts on programming the 

Chapters 21 and 22 discuss the problems of introducing a com- 
puter system into an organization and of managing a computer in- 
stallation. 

This reviewer believes Awad’s book is one of the best introduc- 
tory texts to appear since data processing began to attract the in- 
terest of the general public. Although the topic hardly lends itself 
to easy understanding through an evening’s perusal of a book by 
the fireside, Awad’s text is certainly more nearly clear and more 
suitable for this purpose than many others, 

Ricuarp H. Smita 
California State Polytechnic College 


Pomona, California 


The Outcomes of Counseling and Ps 
ychotherapy: Theory and Re- 
search by Theodore Volsky, Jr., Thomas M, Aten, Warren 
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T. Norman, and Donald P. Hoyt. Minneapolis: University of 
Minnesota Press, 1965. Pp. xi + 209. $5.50. 

Since the establishment of the University of Minnesota Testing 
Bureau in 1932, various groups of educators have devoted their 
time to the evaluation of counseling, psychotherapy, and casework 
which occur in different settings with different clients under the 
sponsorship of a variety of agencies. The present research study 
was conducted in order to examine the psychological interviews in 
terms of their outcomes or effects on the clients’ behavior instead 
of in terms of the counseling process. The researchers in this study 
have attempted to define their problem as being able 1) to identify 
operationally the goals and/or outcomes of counseling and to de- 
termine to what extent such goals and/or outcomes will be realized; 
(2) to develop criterion instruments which will accurately measure 
the clients’ extra-clinical behavior; and (3) to define as clearly as 
possible the nature of the counseling process employed in order to 
achieve the defined objectives. 

The general format of this book consists of ten chapters which 
follow a research outline. A brief summary of the main research 
topics follows: 

In Chapter 1 a general statement of the problems is offered to 
give direction to the conducting of research in counseling and psy- 
chotherapy. 

In Chapters 2 through 5 the general methodological topics are re- 
viewed in terms of how they relate to outcome research in these 
fields. This presentation is by no means complete. However, the 
authors have selected topics which have often been overlooked in 
previous counseling studies or which have received inadequate cov- 
erage in the current literature. Discussed within these four chapters 
are the systematic theory formation, design, techniques in analysis, 
and some associated technical and procedural problems as they re- 
late to outcome research. 

Following this discussion on the methodology and procedures, 
Chapter 6 describes the development of the theoretical framework 
with specification and definition of key variables which were ex- 
amined by the authors. This conceptual framework evolved from 
the counseling or therapeutic processes—objectives to which the 
counselor subscribes as legitimate, desirable, and expected outcomes 
of working with clients. The following five dimensions of client be- 
havior were specified and defined: manifested anxiety, defensive- 
ness, personal-problem-solving ability, motivation to change, and 
perception of counseling. 

In Chapter 7 the authors give the details of the development and 
validation of the criterion measures and classification instruments 
used. 

Chapter 8 outlines the design and analysis as well as the results 
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of the evaluation of this research. Of the null hypotheses tested by 
analysis of covariance, only one could be rejected at the .05 level of 
significance. This rejected null hypothesis indicated that no signifi- 
cant difference existed between the experimental and control groups 
for the problem-solving variate. 

Chapter 9 provides additional descriptive information from two 
other studies. Jewell’s study was conducted because he felt that the 
direction of the desired movement in manifest anxiety, defensive- 
ness, and solving of personal problems could not be pre-specified. 
He took the position that these dimensions should be uniquely speci- 
fied after diagnosis of the individual case. The Vosbeck study in- 
vestigated the unexamined interrelationships among available sets 
of data which might provide direction for subsequent research. This 
study also examined the relationship of client movement on the 
experimental variables to academic outcomes, 

Chapter 10 presents a summary of the conclusions and implica- 
tions of the research as they may relate to future studies, This chap- 
ter is divided into four major issues which the authors hope will 
stimulate future research in the areas of counseling and psycho- 
therapy. The four areas are: clients and the process, criterion prob- 
lems, control methods, and future developments, 

The authors of this book are to be commended for their insight- 
fulness into the areas of counseling, psychotherapy, and casework. 
Of particular significance is the discussion of issues and questions 
which have great importance to researchers conducting studies in 
these areas. The reviewer feels that this book should be useful es- 
pecially for those concerned with developing research techniques 
for future outcome studies and for those concerned with the theo- 
retical work of the neo-behavioral approach to counseling and 
psychotherapy. 

G. Rosert Warp 
University of California, Santa Barbara 


A Social Psychology of Group Processes for Decision-Making by 
Barry E. Collins and Harold Guetzkow. New York: John Wiley 
and Sons, 1964. Pp. x + 254, 

The purpose of this book is to propose a theory of decision- 
making in small face-to-face groups. The proposal emerges induc- 
tively from the synthesis of selected research findings. The authors 
are quick to recognize that such a theory, made up of bits and pieces 


from here and there, is particularly vulnerable to attack, but the 


authors welcome such attacks, for they recognize that the fruitful- 


ness of their formulations will be measured by th i 
. © h 
er are N by other investigators. 4 So iia 
ow did Collins and Guetzkow go about thesizing thei 
theory? First, they abstracted on Separate cards ‘Bios idis which 
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(a) involved rigorously collected and analyzed empirical and quan- 
titative data, and which (b) were reasonably related to decision- 
making in small groups. Second, the authors sorted the cards into 
“topic piles,” each with relatively homogeneous themes and results. 
And third, they formulated propositions based on the grouped ma- 
terial. 

The studies which constituted the bits and pieces are drawn al- 
most entirely from small group research. This body of material is 
broad in the sense that it covers more than decision-making proc- 
esses in small groups—this by necessity, for little material exists 
which is specific to decision-making in small groups. But the biblio- 
graphic material is narrow in the sense that it is restricted to small 
groups; there is no review of the literature in the areas of motiva- 
tion, emotion, learning, or perception, concepts which the authors 
admit as being important to an understanding of group decision- 
making. The authors base their “inductive summary and theory” 
on some 300 small group studies, most of which have been pub- 
lished since 1950. It is natural that the authors rely heavily on the 
results of the Conference Research project in which Guetzkow, 
Donald Marquis, Roger Heyns, and others were involved during 
the years around 1950 at the University of Michigan. 

The theory which emerges is expressed in terms of 62 proposi- 
tions. In addition to its lack of parsimony, the theory sometimes 
lacks definitiveness. For instance, Proposition 2.4-B states that 
“The presence of other individuals may increase the defensiveness 
of the individual; although the effect may be temporary.” Of the 
62 propositions, 10 are of this type. Another six strike the reviewer 
as being trivial, and could readily have been written without bene- 
fit of a review of the literature. For instance, Proposition 10.3 states 
that “Interaction with persons we like and persons who like us will 
produce satisfaction.” Still another eight propositions involve tau- 
tologies. For instance, Proposition 8.1 states that “High power per- 
sons possess more influence,” a proposition which must follow from 
the authors’ characterization of “a high power person as one who 
can influence the behavior of others.” Similarly circular, Proposi- 
tion 3.1 states that “Obstacles originating in the task environment 
directly inhibit the productivity of individual members,” a proposi- 
tion which is hard to distinguish from the definition on page 70 
which states that “A task-environmental obstacle is a particular 
aspect of the total task environment which blocks, inhibits, or 
limits group productivity.” The proposed concept is inferred from 
the behavior it is intended to explain. $ 

The authors utilize two units of analysis (the group and the in- 
dividual in the group) and two classes of independent variables 
(task-environmental and interpersonal). When the individual is 
the unit of analysis, it would seem that variables pertaining to the 
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internal environment (biological variables) would also be impor- 
tant to an understanding of the individual’s decision-making be- 
havior. The absence of this important class of variables is a reflec- 
tion of small group research in general. 

Perhaps the most controversial aspect of the Collins and Guetz- 
kow book is their assertion that the concept of leadership is not 
essential to an understanding of group decision-making, and that 
the explanatory power of this concept is inferior to that of the more 
fundamental concepts (common fate, interpersonal attraction, pres- 
sure toward uniformity, task-environmental and interpersonal ob- 
stacles, task ambiguity, power, satisfaction, and others) of the Col- 
lins-Guetzkow theory. However, the authors are more sympathetic 
to the notion of two leaders within the group, a task-environmental 
leader (idea person) and an interpersonal leader (best-liked per- 
son), a notion consistent with Bales’ orientation. 

With the conference becoming an increasingly pervasive vehicle 
for decision-making, there exists an important applied reason for 
understanding group decision-making processes. Such an under- 
standing will be advanced most rapidly through the conceptualiza- 
tion of the decision-making phenomenon. Collins and Guetzkow 
have presented such a conceptualization based on the integration 
of a vast amount of data from diverse sources, Theirs is a book for 
practitioner and scientist alike. 

EDWARD LEVONIAN 
University of California, Los Angeles 


Integrating the Individual and the Organization by Chris Argyris. 
New York: John Wiley and Sons, Ine., 1964. Pp. x + 330. 

Dr. Argyris has produced a book which should serve as a spring- 
board toward better understanding of organization than has been 
available in the past. Making use of the same basic facts as have 
been used in other books on organization theory, Dr. Argyris not 
only takes into account the high level abstractions usually found 
in organization discussions, but also focuses on the specific fact of 
the individual. Recognition 
up of individuals has been a long time coming. 

This reviewer is especiall 
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There are a few questions which arise even in the first few pages 
of the book. One is the apparent acceptance that physiological and 
psychological energy are different. There is no reason to assume 
that these two types of energy are not identical, nor does the crea- 
tion of “psychological” energy, as such, seem to add to one’s under- 
standing of human behavior. A complication has thereby been 
created which has no essential value in the discussion. Dr. Argyris 
explains the concept of psychological energy on what appears to be 
a verbalization of attitudes which reflect the individual’s percep- 
tion of the environmental situation. This creation of concept often 
results in errors of judgment being made about individual behavior 
in the work situation. If it is recognized for what it is, a reflection 
of attitudes, better understanding results. In the world of behavior, 
these attitudes operating in conflict with the normal output of en- 
ergy not only result in increased resistance, but also create choking 
effects on the physiological energy output. 

To compensate for the questions created by the energy concepts, 
Dr. Argyris makes an excellent presentation of the “self” as being 
an important function of the individual’s behavior. Following the 
understanding of self as being of prime importance, one finds an 
equal emphasis on the factor of self-esteem and the part which it 
may play in the success or failure of the individual within the or- 
ganization. It is further recognized that this individual level of suc- 
cess is reflected in the success or failure of the organization as a 
whole. 

In Chapter 7 the author presents the core of his thinking about 
organization, in the presentation of the “Mix” model. This model is 
illustrated as a series of bi-polar concepts, six of which have been 
spelled out in the book. However, the author recognizes the possi- 
bility that other similar concepts may exist and may be essential 
to the total “Mix” model. The emphasis of the concept of “Mix” is 
on interrelationships of individuals in the organization and on the 
awareness of these interrelationships—all directed toward achiev- 
ing the general objectives of the total organization. It is refreshing 
that this book does not reject any of the accepted organizational 
patterns, many of which have been most successful in practice. It 
does, however, help the reader to recognize that there may be 


many organizational patterns, all of which may be successful under 
certain conditions. It is further recognized that these conditions 
may change and that therefore the pattern of organization may 
also need to adapt by change. f 

All in all, this is a most stimulating book, one which needs to be 
read with an open mind and one which should result in a series of 
studies to verify the hypotheses suggested by Dr. Argyris. There are 
two indices of the value of this book to which the reviewer can 


point. One is that his copy is thoroughly marked emphasizing the 
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important concepts suggested by Dr. Argyris, and the second is that 
he intends to use it as a text book in his courses on management 
theory and organization. 
WALTER V. CLARKE 
Providence, Rhode Island 


Models, Measurement, and Marketing by Peter Langhoff (Editor). 
Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1965. Pp. 
216. 

A book of nine essays first presented by the individual authors at 
the Market Research Council, 1961-62, Models, Measurement, and 
Marketing describes in non-mathematical terms recent advances in 
model-building, simulation, and decision theory for marketing. Al- 
though non-mathematical in exposition, the essays express ideas 
and concepts which utilize mathematics and the scientific method in 
application. 

Peter Langhoff’s essay “The Setting: Some Non-metric Observa- 
tions” sets the framework for the essays which follow. Langhoff 
discusses the rapidly developing field of management science and 
its revolutionary effect upon marketing management and upon 
model building and measurement. 

Philosopher C. West Churchman pursues the pitfalls and prob- 
lems of model devising and applying models. Churchman whose 
concern is the conflict between realism and idealism concludes with 
this basic principle: “a point of view, or a model, is realistic to the 
extent that it can be adequately interpreted, understood, and ac- 
cepted by other points of view” (pp. 37-38). 

Following Churchman are essays by Harold Kuhn on “Mathe- 
matics and Marketing,” by Martin Kenneth Starr on “Computers: 
the Marketing Laboratory,” and by Sebastian B. Littauer on “Fun- 
damental Scientific Aspects of Marketing and the Development of 
Marketing Models.” Each of these essays in its own way sets the 
theoretical framework for model-building, mathematics, and scien- 
tific method. 

Although each essay specifically relates to marketing, the ideas 
and concepts advanced in these five articles also have relevance for 
other areas of the social sciences. For example, the use of computers 
and Monte Carlo techniques could be used for a predictive purpose 
for human behavior in general let alone consumer behavior. Also, 
Professor Kuhn’s outline of a curriculum for mathematically train- 
ing marketing executives is relevant for training hospital or edu- 
cational administrators as well, Computers can bi d for bett 
Mason tana eM e used for better 

s making by any administrator, and an awareness of mathe- 
matics would certainly facilitate their use. 


The remaining four essays in the book descri i 
erib del; 
have been developed for use in marketing, Alfred A. Kucha hs 
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essay “Models for the Budgeting of Advertising” describes three 
areas of application of models: (1) the determination of the ad- 
vertising budget, (2) the selection of advertising media, and (3) the 
evaluation of advertising effectiveness. In “Models of Economic 
Competition,” William J. Baumol approaches marketing decision- 
making from the point-of view of the economist. Baumol discusses 
the use of differential calculus, mathematical programming, and 
game theory for solving business problems. Paul E. Green looks at 
decision and information theory in his essay on “Decision Theory 
in Market Planning and Research.” Finally Guy-Robert Detlefsen 
discusses the problem of putting “Theory into Practice.” 

Perhaps a way to evaluate the book is to look at the objective of 
the writers in putting the book together. On page 4 Langhoff dis- 
cusses the objective of the book: 

A major objective of this book is to expose the reader to the 
basic concepts of mathematical models, information feedback, 
decision theory, stochastic process, and digital computers and to 
sketch the opportunities they may have to offer. 

To determine for whom the book was written, one reads on the 
front of the jacket: “A clear exposition of the implications of the 
new analytical techniques for the work of the marketing executive.” 
However, if the book is designed for the uninformed, the authors in 
their writings often forget this by escaping into the relative security 
of technical jargon without acquainting the reader with the mean- 
ing of these technical words. In addition, only the last 85 pages are 
really concerned with the full implication of these techniques for 
marketing. It is unfortunate that the authors devote relatively more 
of their efforts to the philosophical bases of the techniques and 
relatively less to their applications. As a result of this emphasis on 
theory, the book is deficient of many models which are applicable 
to marketing: queuing models, warehouse location models, inven- 
tory models, and allocation models in general. ‘ 

The objective of the book is noteworthy, for there is need for the 
researcher to inform the practitioner of the fruits of his research 
efforts. But in this instance, the authors spend too much time phi- 
losophizing, and except in the last four essays never do come to 


grips with this problem. 
Kent B. Monrop 
University of Illinois 


Educational Psychology (2nd Edition) by Frederick J. McDonald. 
Belmont, California: Wadsworth Publishing Company, Inc., 
1965. Pp. xviii + 710. $7.50. y La 

McDonald belongs to the new generation of writers in educa- 
tional psychology who are oriented to the learning emphasis and 
who are developing systematic theoretical approaches with a learn- 
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ing model as a structural reference point. McDonald’s point of de- 
parture is a feedback model as a base for instructional strategies. 
Nowhere is the break with the older descriptive and developmental 
orientation more apparent than in the section on Piaget which he 
introduces with these words, “Piaget’s theory ... is not a learning 
theory in the same sense that the concept is used in this book. It is a 
development theory ... (which) must inevitably be related to 
theories of learning. However, it has not been to date, nor is it at all 
obvious how such interrelations can be made . . . Therefore, because 
this book has taken a learning rather than a developmental ap- 
proach to instructional problems, we have not studied Piaget to this 
point” (p, 491). 

The new emphasis on learning models and on the systematic de- 
velopment of teaching strategies based on them is fundamental to 
making teaching an applied science. Each of the models provides 
the teacher with worki g hypotheses that can be tested and modi- 
fied as they are tried in classroom situations, The change is from a 
folklore to a scientific concept approach. 

McDonald’s introductory chapters give the student a working 
knowledge of scientific investigation with the basic statistical tools 
needed to read with comprehension the research literature. Prob- 
ably it is inevitable that the first authors who break with a past 
orientation cannot break completely, or their books will not sell. 
Much of the middle of the book is quite traditionally descriptive. 
It is probable that the next edition, and there probably will be 
many next editions, will pick up some of the points that are trouble- 
some as a strict attempt is made to use the learning model. Although 
the Teviewer is well aware of the fact that curriculum people have 
differentiated for many years between concept formation and gen- 
eralizations, the cognitive difference seems to be unreal. Also, the 
basic relationship between concept formation, developing associa- 
tions, and memory is not clearly developed. 

Another section that probably will be rewritten in the next edi- 
hie : and evaluation. The introductory 
chapter of this section could Probably have assumed that the statis- 


1Q’s without a word about deviation 1Q’ 
standard score approaches seems unfortunate. 
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The section on achievement tests is too thin to affect the teaching 
strategy of any except the most perceptive readers. Teachers make 
decisions about classroom practice on the basis of their knowledge 
of the significance of intelligence and achievement scores much 
more frequently than they do from scores on other standardized 
tests. In a book for teachers these sections need to be carefully de- 
veloped. In spite of flaws, this will be a widely used book and one 
which will grow stronger with successive revisions. The book is 
beautifully written. The style is clear and lucid. The summary sec- 
tions are excellent recapitulations of the contents of the chapter. 
The questions at the end of the chapter are functional for class dis- 
cussion. A glossary at the end of the book is useful to students. 
This is an excellent text book for classes that are ready to move to- 
ward a theoretical approach to learning as a basis for teaching 
practice, 

Jonn A. R. Witson 
University of California, Santa Barbara 


Educational Psychology (Second Edition) by Karl C. Garrison, 
Albert J. Kingston, and Arthur S. McDonald. New York: Ap- 
pleton-Century-Crofts Co., 1964. Pp. xvi + 544. 

In the revision of their text Garrison, Kingston, and McDonald 
present an accomplishment indicative of cooperation, integration, 
and scholarly research. Based upon the recent findings of educa- 
tional psychology, the data attest that “more than peripheral at- 
tention” has been given to current pedagogical problems. The text 
seeks to remedy the divorce apparent in psychological research 
through a culmination of data resulting from sound educational 
theory and practice. il i 

Divided into five major segments the text focuses much attention 
on the learner. The format of the nineteen chapters, documented by 
up-to-date research findings and subdivided according to pertinent 
topic headings, presents a chapter summary, 4 set of discussion 
problems, and related readings for extended knowledge. 

Of particular interest to readers of EDUCATIONAL AND PSYCHOLOGI- 
cau MeasuremeENT is the chapter on “Intelligence,” which treats of 
the usual types of ability tests. Since the Goodenough Draw-A-Man 
scale (p. 75), which appeared in 1926, has been revised and since 
many additional data have been presented, it is unfortunate that no 
indication of this revision is given. (See Dale B. Harris’ Children’s 
Drawings as Measures of Intellectual Maturity: A Revision and 
Extension of the Goodenough Draw-A-Man Test. New York: Har- 
court, Brace and World, 1963. Pp. 367.) Guilford’s “structure of 
the intellect” and findings on creativity do receive some mention. 

In presenting the subtests of the Wechsler Adult Intelligence 
Scale (WAIS), the writers omit the digit symbol test. In describing 
samples of tests the authors cite the 1951 edition of the California 
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Test of Mental Maturity (pp. 334-335). A revision appeared in 1963. 
A discrepancy appears in the use of the Easel Age Scale recom- 
mended for “kindergarten and primary grades” (p. 354). More 
specifically it is for pupils in kindergarten and grade one according 
to the test manual. Rothney is cited as the source of the objectives 
of evaluation (p. 344). They are also to be found in many other 
early sources—especially in the monumental works of Ralph Tyler 
and his associates. These are minor points when the text is evalu- 
ated in toto. 

The chapters are not overly general but specific enough to rec- 
ommend their serious reading and application. Treatment of pupil 
evaluation embraces test devices both standardized and teacher- 
made, methods of studying child behavior, diagnostic and reme- 
dial treatment of learning difficulties. Discussion of pupil person- 
ality along with teacher behavior challenges the reader. 

With the up-to-date bibliography, a list of films, and problems 
for discussion, the text fulfills a definite role in undergraduate and 
graduate classes in educational psychology. 

BISTER JOSEPHINA, C.8J. 
Boston College 


Exemplars of Teaching Method by Harry $. Broudy and John R. 
beak Chicago: Rand McNally and Company, 1965. Pp. vii + 
‘To compensate for the pedestrian nature of many textbooks, the 
wise teacher will develop a list of outside library readings or other 
supplementary works, in the hope that at least one of them will 
teach” the student and stimulate him to further inquiry. A note- 
worthy addition to the list is Harry $. Broudy and John R. Pal- 
mer’s Exemplars of Teaching Method. Many readers will perhaps 
be familiar with Broudy’s original essay as it appeared in the 
Handbook of Research on Teaching, edited by N. L. Gage in 1963. 
The present volume is an enlarged and rewritten version of the 
original essay. It makes no claim to completeness: its purpose is not 
that of a textbook in the history of education. The book is intended 
rather as “a supplement to systematic textbooks in the history of 
education and as a source of background materials in administra- 
T a eee methods, and other standard courses in educa- 


The book is arranged so as to give one or mor i 
i e eminent examples 
(“exemplars”) of ten different methods and philosophies of tench! 
a ranging from the ancient rhetoricians to modern progressivists. 
( e reader will encounter such enticing chapter headings as 
“Teaching Courtly Barbarians: Alcuin,” and “Dialectical Ga, d i 
ing: Froebel.”) 4 eestor 


Once the reader identifies with one or more of the exemplars, he 
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is forced to come to grips with topics that take him far beyond an 
isolated review of methods used by these teachers. The reader will 
find himself wrestling with the matters of ultimate objectives, short- 
term versus long-term goals, the place of vocational, intellectual 
and physical activities, organization and administration, and a 
vast range of educational problems. One of the strengths of this 
approach is that through its sympathetic exposition of diverse 
teaching methods it provides the student with a whole spectrum of 
possibilities—possibilities which would otherwise probably never 
suggest themselves. There is more than one way to carry out the 
process called education, and it helps to know the advantages and 
disadvantages of each. 

An exposure to this book should cure anyone of the naive view 
that learning is solely a cognitive matter. The student who uncon- 
sciously imitates his teacher is learning on a non-cognitive level 
how to become a different kind of person. As the authors somewhat 
rhetorically ask, does the teacher “also serve as an image of a value 
system that, aesthetically apprehended, conveys to the student cer- 
tain models that invite imitation?” There is no doubt that he does. 
It might even be the case that the more important things, such as 
values and attitudes, are taught in this “aesthetic” way. 

Let us now turn our attention to several weaknesses or limita- 
tions of Broudy and Palmer's approach. The most serious objection 
might be the lack of adequate coverage. The book does not pretend 
to offer more than a sampling of exemplars; but even a sampling, to 
be at all representative, must not limit itself to the western world. 
What is wanted is a fair selection of models who represent the ma- 
jor traditions in teaching methods. And it just so happens that 
there are teaching methodologies in the East that have no parallel 
in the West. Therefore, to limit oneself to the Greco-Roman, Ju- 
diac-Christian traditions is to restrict one’s educational horizons. 
There is nothing in the book to even suggest that some of the most 
interesting methodologies are to be found in the Orient. 

Closely connected with the above criticism is the question of the 
adequacy of the four criteria by which the authors selected exem- 
plars for inclusion in their book. The four criteria are: First, each 
must have had a “distinctive style of teaching”; second, each must 
have been “identified with education and formal instruction”; 
third, besides having a distinctive method, these men also “illus- 
trate a way of life, a system of values, and a response to a cultural 
challenge”; fourth, they have been “interested in methods and 
often have been articulate about them. They belonged to the tribe 
of professional teachers, not only in the sense that teaching was 
their major life occupation, but also in the sense that the process of 
teaching and learning seemed to them worthy of serious and spe- 
cialized study.” Religious leaders who were teachers in the wider 
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sense of the term, such as Jesus or Buddha, were excluded on the 
grounds that “their goals transcended learning and instruction” 
and that ‘they were not conspicuously interested in the methods by 
which they achieved their results.” 

It is difficult to see how the Scholastics were selected for inclu- 
sion, since their goals surely transcend learning and instruction. 
Be this as it may, a more serious problem is the exclusion of religi- 
ous leaders on the grounds that they were not especially interested 
in the methods used to achieve their goals. But is this true? It is not 
easy to see how Buddha could be excluded on these grounds, for he 
systematically expounded and personally practiced one of the most 
outstanding educational techniques known to man. One can hardly 
quarrel with the authors’ choice to limit the book to secular educa- 
tion if for no other reason than that of space and manageability. 
But the criteria stated fail to delineate accurately between religious 
leaders and educators. Perhaps this failure is a clue to the possibil- 
ity that there is no sharp division between the two. 

WILLIAM GEORGIADES 
University of Southern California 


Culture in American Education: Anthropological Approaches to 
Minority and Dominant Groups in the Schools by Ruth Landes. 
New York: John Wiley & Sons, 1965. Pp. vi + 330. 

Although the nation’s attention has been directed by. politicians 
and educators alike to America’s “disadvantaged,” teachers in the 
field are frequently perplexed as to how they should personally deal 
with problems resulting from the diverse subcultures which exist in 
America. Professor Landes’ book is therefore timely and also im- 
portant as an example of cooperation between anthropology and 
education. 

The book describes the Anthropology and Education program 
conducted at the Claremont Graduate School from 1959-61. As 
director of the project Dr. Landes worked with the Claremont edu- 
cation faculty to help Southern California public school teachers 
understand and to use culture concepts and methods affecting pub- 
lic education. Three main avenues were used: (a) graduate semi- 
nars in anthropology and education, (b) classroom observation and 
discussion with faculty member 
secondary schools and some juni 
the development of an elementa: 
cipals and teachers in a large city of Southern California. 

The location of the study is significant, for, as the author indi- 
cates, California is a microcosm of the world’s races 


Perhaps the most heartening aspect of the difficult task assumed 
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by the project staff was that teachers learned to understand better 
themselves and their students by applying basic, simple social sci- 
ence research procedures. Some of the procedures used were: obser- 
vations of individuals and groups, “open-ended” interviews of in- 
formants, and more specific questionnaires. Data collected were 
shared with fellow students and professors. An immediate objective 
of the project was for teachers to understand that their own family 
cultures, like the minorities they were studying, varied greatly. 
This was accomplished by having each teacher study his own back- 
ground in a systematic manner. 
| Results of the Anthropology and Education program are dis- 
a cussed in Chapter Seven, “Teacher Research in Culture and Educa- 
a 4 tion.” Sections of teacher reports are quoted at length. Thus are 
demonstrated new understandings gained by teachers as a result of 


ke the project’s treatment. Professor Landes, in closing, emphasizes the 
ZA need for effective organization of entire school systems to achieve 
\e desired results with active associations between parents and school 
id personnel of primary importance. 

d Although the book deals specifically with the Claremont Project, 
Vg it might well be used as a supplementary book in any Social Foun- 


dations of Education course. Chapter Three, “Culture Concepts and 
i Methods for Teachers,” is an excellent general introduction to cul- 
ee ture and its relationship to education. The author brings to this 
a chapter a rich background from a life devoted to the study of an- 
thropology. 


DALE L. BRUBAKER 
University of California, Santa Barbara 


Handwriting in Psychological Interpretations by Arthur G. Holt. 
Springfield, Illinois: Charles C. Thomas, 1965. Pp. xiii + 262. 
$10.50. 

The author, a Doctor of Jurisprudence, has devoted forty years 
of his life gathering the information he offers in this work, and 
states his case as follows: 

There is a method to acquire extensive knowledge of self and 

also of others without personal contact with them, which elimi- 

R nates the danger of being unduly influenced by likes and dis- 

| likes. It is the science of handwriting interpretation as demon- 
strated in this book. 

Dr. Holt presents in various styles of writing “standard letters of 
domestic and foreign alphabets” and comments upon each letter as 
to the character traits and personality tendencies of the writer. 
Preliminaries are also offered for further analysis along with other 
pertinent information. 

Chapter 2, entitled “The Science of Graphology and the Under- 
lying Theory of This Approach,” contains a brief history of the 
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__ » subject. The reader is told: “In America many research materials 
- are‘available, but very few are written in textbook form.” Also, the 
` reader is ‘appraised of the fact that West German and French Uni- 
4 [7 yersities include graphology in a “clinical or applied psycholagy 
© “curriculum.” ~ 
What the author does do effectively is to state his belief clearly 
and to present descriptive information. What is needed now, how- 
ever, is carefully researched experimental data. Perhaps another 
‘volume containing material of this nature will*follow. 

! ARTHUR LERNER 

# : Los Angeles City College 


e 


The Underachiever: Challenges and Guidelines by ©. Burleigh 
Wellington ‘and Jean Wellington. Chicago: Rand McNally & 
Company, 1965. Pp. x + 122. $1.75. 

This paperback contains a wealth of information about the uñ- 
derachiever, including existing research and background material 
for identification, selection, and assessment problems in the field. 
The author believes that through an understanding of the similari- 
ties among underachievers sound helping programs can be estab-- 
lished. Although the book is essentially oriented to the school situ- 
ation, much of the information may also be helpful to the parents of 
underachievérs. 
| The Underachiever: Challenges and Guidelines, which makes a 
serious attempt to explain research findings in a non-technical 
manner, will prove to be a handy reference on many practical and 
theoretical levels. Its presence is a vital contribution to a most 
complex phenomenon of the educational experience, 

. ÅRTHUR LERNER 
Los Angeles City College 
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MULTIPLE FACTOR "ANALYSIS ral 
IN TERMS OF WEIGHTED REGRESSION! . 


JOHN M. BUTLER anv L. HARMON HOOK 
The University of Chicago 
Bh ih 
4 A 
Somm basic problems in factor analysis have been clarified in the 
last two decades in important papers by Guttman (1940; 1953, , 
1956), Lawley (1940), Rao (1955), Harris (1962, 1963) and Kai- 
ser? (1958, 1963). In particular their work clarifies the “concept of 
what Thurstone (1947) called unique variance and the communal- 
ity concept or closely related concepts. It is, however, the very 
centering upon the communality problem and closely related "prob- 
lems which constitutes a consequential departure from multiple 
factor analysis as envisaged by Thurstone. As Thurstone envisaged , 
it, factor analysis should have as a main purpose specification ‘of | 
the domain or population of factors. For any given, sample of tests 
it might be that population common factors are in varying de- 
grees well represented, poorly represented or not represented at all. 
Common factors poorly and fractionally represented in the tests 
- were called specific factors by Thurstone, and he stated that such 
-factors may become common factors in a test battery when a given 
battery is augmented by additional tests. It follows from what 
Thurstone has said about specific factors that the factorial domain 
cannot be specified until a test battery well represents that domain. 
Since in practice the domain is known through a set of related 
measurements, it seems somewhat beside the point to estimate 


1 The views presented here are an outgrowth of research supported by re- 
search grants USPHS MH 04609-04, USPHS MH 10131-01, NIMH, U. 8. 
Public Health Service. 

2 The writers have been considerably influenced by the papers of these au- 
thors. However, the influence is so pervasive that little attempt will be made 
to particularize the indebtedness beyond the listing of references. 
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population communalities or uniquenesses unless one has good 
reason to believe that a test battery is representative of a factorial 
domain. In this case there would be no specific variance in the test 
battery. Unfortunately, zero specific variance cannot insure that a 
factorial domain is well represented in a test battery because of the 
possibility that some factors may not be represented at all. For 
these reasons the discussion here will be confined to the sample of 
tests, and the communality problem and the problem of the num- 
ber of factors will be considered in relation to sets of vectors which 
will be designated as unique vectors and specific vectors or factors. 

Harris (1963) and Guttman (1956) have obtained uniqueness 
values, v,*, such that 


v? = bi} [eT ) 
where r/ is a row vector composed of the diagonal elements of R-t, 
the inverse of the correlation matrix, and (r/*)? is the square matrix 
with elements consisting of the squares of the corresponding ele- 
ments of R. These uniqueness values have the disadvantage that 
they may be greater than maximum uniqueness values ; further- ` 
more, they may assume negative values (Harris, 1963, p. 147). Evi- 
dently, then, the v do not conform to the fundamental require- 
ments that uniqueness values be positive and that communalities 
have a maximum value of unity. 


The Factor Model 


The approach to be taken here will be based upon regarding the 
uniqueness-communality problem as a problem in weighting a re- 
gression equation, This amounts, as will be shown later, to weight- 
ing maximum (and always positive) uniqueness values by positive 
values between zero and unity. 

Let S be the n X t matrix of normalized standard scores” 
(Thurstone, 1947, p. 367) for n tests and ¢ subjects with each test 
vector being orthogonal to no more than n — 2 test vectors, Then 
let F be an n X n matrix such that 


SS’ = FF' =R (2) 


where R is the correlation matrix. Then F is a matrix representa- 
tion of the test vectors in an arbitrary orthogonal basis such that 
the row vectors, as representations of the test vectors have unit 
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length. F, being square and having rank n, has an inverse, F+, so 


FF = 1. (3) 
Equation (3) shows that the identity matrix represents the per- 
pendicular projections of the column vectors of F- upon the unit 
length row vectors represented in F. Since each column vector of 
F- is orthogonal to every row vector of F but one, the column vec- 
tors of F- are the unique vectors for the sample of tests. Normalizing 
the column vectors by post-multiplying F- by the diagonal matrix 
of normalizing constants, D, gives 
FF"D = D. (4) 
Equation (4) shows that D is the matrix of intercorrelations of the 
unique and test vectors. 

Given the elements of D as the correlations between the test 
vectors and the unique vectors it has been shown (Guttman, 1940) 
that 

I-D =È (5) 
where & is the diagonal matrix of multiple correlations between 
each test and the remaining n — 1 tests. R? is the diagonal ma- 
trix of communalities corresponding to the test vectors when the 
unique vectors have unit length. These are the minimum communal- 
ities (Guttman, 1953, 1956). It should be noted also that the ele- 
ments of D? are maximum values because the unique vectors cannot 
be taken as having length greater than unity, The maximum 
uniquenesses and minimum communalities are thus functions of 
cosines of angles. 

In multiple factor analysis the test vectors may be considered to 
be the vector sum of the unique vectors and what will here be 
called “common” vectors. The equation for the vector sum, ex- 
pressed in terms of F is 

F=C+KrFr™ (6) 
where C is the matrix of common vectors and KF- is the matrix 
of unique vectors with the unique vectors being adjusted for length 
by the diagonal matrix K with elements k; such that 


k; < d? 
where d;? is the i-th element of D?. The row vectors of F, the matrix 
of test vectors; C, the matrix of common vector s, and KF’, the 
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matrix of unique vectors, are all expressed in the same arbitrary 
orthogonal basis. 

Now let K = D? and C = WF where W is a matrix of weights 
to be applied to F to obtain C. Then (6) becomes, after some re- 
arranging, 


C=WF=F-— DF", 7) 
Post-multiplying (7) by F-1 gives 


W =I- DFF”, (8) 

It is known (Guttman, 1940; Kaiser, 1963) that W is the matrix 
of multiple regression coefficients to be used for predicting a given 
test score from the remaining n — 1 tests. Thus the common vectors 
are the predicted vectors when a least squares criterion is used. 
Note that the diagonal entries of D2 are the inverses of the diago- 
nals of FF- so the diagonals of W are zero. What (8) shows is 
that the use of D? for uniqueness values is equivalent to regarding 
factor analysis in multiple regression terms (D? is the error variance 
in the regression equation). 

C may also be obtained quite simply by subtraction. However, 
when this is done K is not specified. 

F may be transformed to GH1⁄2 where G is an orthogonal matrix 
and H’/? is a diagonal matrix such that 


GHG =R. (9) 
Then GH1/ is a principal axis factor matrix for the correlations 


and the matrix of common vectors can be written in principal axis 
terms as follows: 


C = GH” — DGH”, (10) 
Pre-multiplying C by D- gives 


D"C = D"GH'? — DRH", (11) 


Since GH? and GH? Tepresent corresponding test and unique 
vectors, the inner products of the vectors, D;"G;H’” and D,G,H7”” 
are unity, and those of D;"G,H'/? and D@:H-” are zero. 

From the right side of (11) it can be seen that corresponding 
to D"GH™”, the weighted Principal axis matrix, is another principal 
axis matrix, HP’, and corresponding to DGH~? is another principal 
axis matrix EP". EP? ean be obtained from D~'GHG'D™. 
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(11) can then be rewritten in terms of EP’ and EP~*”” as follows: 
DE = EP? Ve Ep"? 
= EP? — Pp”), 
‘The covariance matrix for the weighted common vectors is 
E(P” — PYE. (13) 
It is important to note that factoring the covariance matrix of the 
weighted common vectors (13) to obtain the principal axis factor 
matrix will not give E(P? — P™?) except when P > 1. Thus 
a proper factoring of the covariance matrix of the weighted common 
vectors is given by E(P'? — P~™) and not by E[(P’? — P~*)?*]}}? 
except when all values of P? are greater than unity.* 

The weighted test vectors and common vectors can be returned 
to their original scale by pre-multiplication as follows: DEP” is 
a factor matrix for the test vectors and DE(P™? — P~*”) is a factor 
matrix for the common vectors. 

The column vectors of DEP™ and DE(P™’ — P~“”) are the 
same except for scale for those values of P’” above unity but not 
for those values of P’” less than unity. This conclusion may be 
restated as follows: Given the covariance matrices of the weighted 
common vectors and the weighted test vectors and applying principal 
axis factoring routines results in factor matrices congruent to J, 
whereas obtaining H(P’? — P~%) directly from EP” gives a 
factor matrix such that E(P? — P~’”*)E’ is congruent to diag. 
(Ip, —In-p) where p is the index integer. Thus only the first p factors 
are common factors in the total common factor space of the sample 
of tests. The index integer p is the same number as Guttman’s 


strongest lower bound for the number of factors in the population. 
The preceding development is based on a regression point of 


view in which D? represents unpredictability. The logic of this 
point of view requires that the entire true variance be common 
factor variance while the logic of the multiple factor analytic point 
of view enunciated by Thurstone requires that the elements of D? 
represent upper bounds to the uniquenesses, being the uniquenesses 
only when the variance due to specific vectors is zero. Now specific 


(12) 


8 Guttman (1953) and Kaiser (1963) define the partial image vectors in 
terms of the matrix DE[(P¥* — P-*)*]**, so common vectors and partial image 
vectors are not in general identical. 
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vector variance is common factor variance from factors poorly 
and fractionally represented in the tests of the battery (Thurstone, 
1947, p. 75), so poorly and fractionally that a regression approach : 
counts it as error variance since the regression model, which is ad- ` 
dressed to prediction of each test from the remaining tests, is con- 
cerned only with maximizing predictability. Specific variance is a 
sample phenomenon and is relative to a particular battery of tests. 
What is specific variance in a given test battery may become com- 
mon factor variance when the battery of tests is augmented by 
other tests. It follows that when a sample of tests does reflect a 
Population of factors adequately, there will be no specific variance. 
On the other hand, it does not follow that the test battery ade- 
quately reflects the domain when there is no specific variance be- 
cause the battery may not represent some of the factors in the do- 
main at all. Guttman (1953, 1956) has stressed that the side 
entries of the inverse of the actual correlation matrix must be 
“small” or zero if the data are to conform to the factor analytic 
model or if the test battery is to be considered representative of the 
universe of content (or of the population of factors). The require- 
ment that the inverse of the correlation matrix be diagonal or 
“nearly” diagonal and the assertion that the test battery then ade- 
quately taps the population of factors leads logically to the re- 
quirement that the squared multiple correlation coefficient be used 
at least initially as a communality estimate. Then using additional 
tests to augment a test battery can only lead to changes in squared 
multiple correlations occasioned by error variance. 

Now all of this is true enough if one knows or has good reason to 
believe that the battery of tests adequately reflects the population 
of factors, However, when one is using factor analysis as an ex- 
ploratory device in the manner of Thurstone, one does not have 


tu , bolts, and light bulbs. 
validity corresponds to actual validity 
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„whereas for psychological tests a test with high face validity may 
not at all perform the task for which it was designed. Since tests 
are constructed on the basis of psychological hypotheses, the “pop- 
ulation” of tests may well depend only on the fertility of the hu- 
man mind and populations may be continually redefined. That 
additional test construction will not result in new factors in any 
domain so far investigated seems fairly unlikely. It seems, there- 
fore, that in the absence of compelling reasons for believing that a 
test battery is representative of a definite population of tests (or 
of factors) it is somewhat feckless to think of population estimates 
of communalities or of uniquenesses. The situations in which the 
population of factors is known or in which the sample of tests is 
representative are, to understate the case, rare. 


Sample Specific Factors 


A sample specific vector must be uncorrelated with the remaining 
n — 1 specific vectors, The i-th specific vector must have as high 
correlations as possible with the remaining test and unique vectors. 
The i-th specific vector can have zero correlations with the j-th test 
and unique vectors only when the inverse of the correlation matrix 
is diagonal. Letting D; denote the diagonal matrix of squared cor- 
relations of the i-th specific and the i-th unique vectors we have 
the proportion of the i-th unique vector attributable to specific fac- 
tor variance. The squared correlations of the diagonal matrix D? 
give the proportions of the test variances which are unique variance, 
and the matrix product DD? gives the proportion of the total test 
variances attributable to specific factor variance. That is, they give 
the proportion of common factor variance poorly and fractionally 
represented in the zero-order correlations of the correlation matrix. 

Given D; the factor matrix for the unique vectors can be weighted 
by D} to get 


F=C+D/D°F'",. (18) 
(18) expresses the regression equation weighted by the proportion 
of the unpredictability which is occasioned by factors poorly repre- 
sented in the tests. Since the elements of D; are correlation coeffi- 
cients, the elements of DD? can be unity only when D? is an iden- 
tity matrix and D? will be an identity matrix only when R> is 
diagonal. 
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The elements of the weighted uniqueness matrix have the follow- 
ing desirable properties not possessed by the Guttman-Harris 
uniqueness values (Harris, 1963) and required by the multiple 
factor analytic model: 


1. Elements of D,7D? are always positive. 
2. Elements of D,7D? are always in the range from zero to d; where 
d; is a diagonal element of D? and0 < dê < 1. 


It seems likely that the elements of D?D? will each be greater 
than any “true” uniqueness values. 

The matrix of weighted maximum uniqueness values has not been 
specified in terms which admit of computation. However, the set of 
elements for D? can be specified as a consequence of noting that 
the i-th specific vector must have higher correlations with the i-th 
test and unique vectors than with the j-th test and unique vectors. 
Consider the matrix, 


L(F + DF’) (14) 
where L is a diagonal matrix of normalizing constants for the row 
vectors of F + DF’, and the matrices of correlations 


L(F + DF'")F’ = L(R + D) (15) 
and 


L(F + DF’")F"D = L(DR™D + D). (16) 
Since the elements of L are less than unity, it is clear that the cor- 
respondingly placed diagonal correlations of (15) and (16) are 
equal and that the off-diagonal correlations of (15) and (16) are 
less than the correspondingly placed correlations in R and DR“D. 
Also the off-diagonal elements of the right side of (16) are in gen- 
eral less than those on the right side of (15). These facts suggest 
that the row vectors of (14) or of suitable linear combinations of 
them can be used to obtain suitable values for DŽ; i.e., the row 
Vectors of (14) or suitable combinations of them can appropriately 
be viewed as specific vectors. Although the diagonal correlations of 
(16) might be used as values of D; it is generally true that the row 
vectors of (14) are not orthogonal even though (14) more nearly 
approximates an orthogonal matrix than does F or DF’, However, 
an orthogonal matrix with row vectors possessing the salient char- 
acteristics of the specific vectors may be obtained by iterating on 
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the process employed in obtaining (14). The iteration process is 
described below.* 
Let Fo be the n X n matrix with test vectors as rows such that 


FF,’ = Ro 
the correlation matrix. Then D,F’y, where Dy is the diagonal matrix 
of normalizing constants for the row vectors of F's}, is a factor 
matrix for the matrix of intercorrelations of the unique vectors and 


FFD, = D, (17) 
is the matrix of correlations of the test vectors and the unique 
vectors. 

The unique vectors as rows of D,F’)~ are represented in the same 
orthogonal basis as the test vectors. The centroid vectors formed by 
summing corresponding test and unique vectors comprise the row 
vectors of G, where 


G = F, + D.F’. (18) 
Normalizing the rows of (18) by the diagonal matrix, Lo, gives 
F, = LF, +L,D.F'.". (19) 
The matrix of correlations of the row vectors of (19) is 
Rı = FF’. (20) 


Rı more nearly approximates an identity matrix than does Ro since 
the row vectors of F, are the unit centroid vectors formed from 
corresponding test and unique vectors. Since the rows of F are unit 
length vectors, Fı more nearly approximates an orthogonal matrix 
than does Fo. F, is non-singular and has an inverse. Iterating upon 
the procedure used on Fy gives 


F, = L,F, + L,D,F',* (21) 
where L4 is the diagonal matrix of normalizing constants for the 
rows of F, + DF” and D; is the diagonal matrix of normalizing 
constants for the rows of F’;". 

Since 


FF’ =], (22) 


4 Although there may be a theoretically better solution based on using latent 
vectors rather than centroid vectors, the present approach seems satisfactory 
and should in any event yield results virtually the same as those obtained 
from using latent vectors. 
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the row vectors of F bear the same relationship to the row vectors 
of F’,~ as those of F, bear to F’,-1. 

Hook (Butler, et al., 1963, App. B) has shown that the sequence 
of matrices 


Ra; Ry, Rz, per RB; (23) 
formed as above converges to the identity matrix. This convergence 
in turn implies that the sequence of matrices, F; with unit length 


row vectors, all vectors being expressed in the same orthogonal 
basis, 


For Fi, Fa, +++ F; (24) 


converges to an orthogonal matrix (but not to an identity matrix). 
The matrices 


FF, FFs, tee , FF, (25) 
give the inner products of the column vectors of the F,. FF; ap- 
proximates an identity matrix as closely as one wishes. 

The matrix of correlations of the unique vectors and the vectors 
of the final orthogonal basis of unit centroid vectors is 


P, = DF P’, (26) 

The i-th unit centroid vector represented by the i-th row vector 
of F is determined entirely by the i-th row vector of Fo, the test 
Vectors in an arbitrary orthogonal basis, and the i-th row vector of 
D.F’, the matrix of unique vectors expressed in the same orthog- 
onal basis. Note, however, that each unique vector is actually de- 


termined by all of the n — 1 test vectors except its corresponding 
test vector. 


i-th test and unique vectors and 
remaining n — 1 test and unique 
tors of F, comprise a collection of 
ating orthogonality than the collec- 
but, as with the unique vectors, the i-th cen- 
similar to the i-th test vector and dissimilar to 


vectors, The unit centroid vec 
vectors more nearly approxim: 
tion of unique vectors, 
troid vector will be 
the j-th test vector, 


The diagonal matrix, D, is a diagonal matrix of correlations for 
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weighting unique vectors and the diagonal values of Ê; are the cor- 
relations of the unique vectors with their corresponding specific 
vectors. If d; is high, then the corresponding diagonal value of F, 
will be high. Conversely, if d; is low, then the corresponding diagonal 
element of Fy will be low. The matrix of diagonal values of Fy; 

diag. F, = D; (27) 
of correlations of uniqueness vectors with corresponding specific 
vectors gives the similarity of each uniqueness vector with its cor- 
responding specific vector. It should be noted that, from their man- 
ner of formation, the diagonal values of F will be equal to or greater 
than the off-diagonal values. 

The use of D} as a weight matrix additional to D? utilizes the 
fact that starting with F and F, and taking successive sets of unit 
centroid vectors simultaneously best fitting corresponding test and 
unique vectors, a final set of orthonormal specifie vectors is ob- 
tained having cosines as high as possible with corresponding test 
and unique vectors and cosines as low as possible with the remain- 
ing n — 1 test and unique vectors. 

The equation corresponding to the modification of (10) by D~ is 


D DiC =D D 0u E D,DGH-””’. (28) 
The corresponding row vectors of the matrices on the right of (28) 
have unit inner products and the non-corresponding row vectors 
have zero inner products. The principal axis matrices corresponding 
to the matrices of (28) may, as before, be written in the form 


PEATE (29) 
EP”, (30) 

and 
EP’, (81) 


although the latent vectors and latent roots are different from those 
corresponding to weighting by D? only. 
Once D,7D* has been obtained for use as a weight matrix, the 
weighted correlation matrix 
D;"D“RD™“D;* (82) 
should be factored by a principal axis factoring to obtain EP”. 
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No more than p factors should be extracted where p is the index 
integer corresponding to E(P™? — P~)H’. This amounts to taking 
as the principal axis factor matrix of the common vectors those rows 
of E(P™? — P~”) for which the values of (P? — P7™?) are positive. 
This gives the maximum number of factors in the sample common 
factor space for a given D,D’, 


Number of Factors To Be Taken from the Sample 


The practice of taking a much greater number of factors than has 
been usual among factor analysts has been advocated by Guttman 
-(1953), Harris (1963) and Kaiser (1963). There is reason, however, 
for viewing this practice, which seems to be becoming conventional, 
with some reserve. First: A consideration of the number of degrees 
of freedom available for testing the number of factors shows that 
the number available is a function of the number of tests and the 
number of factors extracted. Suppose the number of factors is taken 
as p, the number of positive values of (P2_P4/2). This is the 
index number of diag. (Ip, —In-p) with index integer P, and n—p = 
Tt, the signature integer, where n is the number of tests. Then the 
number of degrees of freedom available for testing the number of 
factors (Rao, 1955) expressed in terms of the index and signature is 


af. = f° — (n + p)]/2 (88) 
which shows that the inequality 


P>n+tp (34) 
must hold if degrees of freedom are to be available for testing the 
hypothesis that there are at least p common factors in the popula- 
tion. With an n of 12, a p of 8, and an 72 of 16, n + p is 20. Hence 
the maximum testable number of factors for a battery of 12 tests 
is 7, even though Guttman’s strongest lower bound for the number 
of factors in the population may exceed 7. Thus the practice of ex- 
tracting p factors cannot be applied without consideration of the 


the practice. 

Second: Consider the inner products of the weighted test vectors, 
pas En in the rows of EP12, and the row vectors of H(P1/2 — 
P=1/2)-1, These inner products are the elements of 
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EP(P — TE’. (85) 
The row vectors of Æ (P1/2 — P-1/2)-4 should have small inner prod- 
ucts with the test vectors of EP12, Table 1 shows that these inner 
products will be small only when P; = 2. When this is so, the latent 
roots of (35) decrease from 2 to 1.01 as the latest roots, Py, in- 
crease from 2 to 100. As P; goes from 2 to 1, however, the latent 
roots of (33) go from 2 to 101. Thus as P, decreases below 2, the 
elements of EP(P — I)“ increase rapidly in magnitude. 


TABLE 1 
Values of Various Latent Roots 


r 
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This implies a decreasing similarity of the weighted test vectors to 
the weighted common vectors as the P, decrease. It also implies, 
that when Prt are below unity, rotation to simple structure will 
eventuate in null or uninterpretable factors, the number of such fac- 
tors being the number of latent roots, Pr*, with values below unity. 
This is shown directly by the matrix of inner products of the 
weighted test vectors and the weighted common vectors, 


EPP” — P-”)E! = E(P — DE’. (36) 
The greater the (P;), the greater the off-diagonal elements of (36) 


will be. 
It should be noted that pre- and post-multiplying (36) by DD, 


and (DD,)’ gives 
DD,D;"D“RD"D;"D;D — D?D? =R — D?D’, (87) 
the correlation matrix with the communalities in the diagonals, 
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Since E(P — I)H’, like E(P? — P~'”)E’, is congruent to diag. 
(Z,, —I,-»), and since the diagonal values of (P— 7)" > (PAPA), 
the inner products of 


(EP (P -—D“E,> (EP (P — PAE’. 


Hence, E(P — I)*is a better factor matrix than E(P? — p~” *) be- 
cause of higher inner products with the original variables and, further- 
more, using E(P — I)? is equivalent to factoring the correlation 
matrix with communalities in the diagonals for D;"D"E(P — 1)? 
is a weighted principal axis factor matrix for the correlation matrix 
with communalities in the diagonals (Harris, 1962, 1963). 

The results of Table 1 suggest that the number of factors to be 
extracted from the correlation matrix should be restricted to the 
number of values of P, the diagonal matrix of latent roots of the 
weighted correlation matrix, 


D D RDD, (38) 
equal to or above 2. The maximum number of factors will be the 
number of values, P;, above unity. Further (34) and (35) indicate 
the general inapplicability of the procedure of extracting p factors. 


Finally, and crucially, observe that R can be written as fol- 
lows: 


R= FGF —T417 (39) 
and 


R=FG—DF +1 (40) 
so we have R and R — J expressed in terms of a common set of 
unit latent vectors and the latent roots of G and G — I. 

With the restrictions earlier placed on the correlation matrix, 
rank n and non-zero off-diagonal correlations in each row, there 
will be at least one latent Toot below unity in R. It follows that 
R — Tis in the same congruence class as J where 


J = Diag. (I,, —I,_,) (41) 
with index integer r. Let X be any di ix wi iti 

: y diagonal matrix with positive 
constants. Then X(R — 1)X is also a member of the canonical 
class J. Thus the number of factors determined solely by the side 
entries of the correlation matrix is the index integer r of J, which is 


BUTLER AND HOOK 559 


also the number of latent roots of the correlation matrix above 
unity. 


Steps in Factoring 


For practical purposes it is recommended that: 
1, The number of factors be taken as the number of latent roots 
of the correlation matrix above unity. 
. Use D;DE (P — I)}/? as the final prerotation factor matrix. 
3. Rotate the factors of (2) to an orthogonal approximation to 
simple structure (Kaiser, 1963). 

4. If s essentially null factors appear upon rotation take r — s 
factors of D;DE(P — I)}/ and re-rotate. Continue the 
process until one essentially null factor appears after rotation. 
If an oblique simple structure solution is desired, rotate only 
after first obtaining the orthogonal approximation to simple 
structure (Kaiser, 1963). 


bs 


ae 


These recommendations bearing on rotation were written on the 
assumption of use of analytical means of rotation. If graphical 
rotation is resorted to, then direct rotation to oblique simple struc- 
ture can be accomplished. Simple structure should be the final 
arbiter of the number of factors in a sample, not statistical tests. 


Final Factor Matrices 


The correlations between the first p factors of 


EP” 2 
and 


EP-D” 


are unity. Therefore, although E(P — I)” is equivalent to a factoring 
of the correlation matrix with communalities in the diagonals, one 
could elect to use EP™? as the matrix from which to rotate the 
simple structure for both HP” and E(P — I)” represent different 
weightings of the correlation matrix by functions of the communalities, 

There is reason to believe that for a given test battery HP? 
the weighted test vectors might have a somewhat better simple 
structure than the weighted test vectors of E(P — I)'”*. The latter 
matrix is obtained by subtracting unity from each latent root of P. 
The result is that the last latent roots of P — I become dispropor- 


560 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


tionately small in relation to the first principal axis when compared 
with the latent roots of P. For example, 


r-1 ro 
p,/ SP, <P.-1/ Ep, 


where 1 ¥ j and r is the number of factors. This means that in 
using an analytic solution such as normal varimax, the simple 
structure factor with the highest cosine with the first principal 
axis is likely to have higher factor loadings for E(P — J)” than 
for EP’. It should be remembered, however, that the number 
of factors to be extracted must be the number of latent roots of 
the correlation matrix above unity if one wishes to restrict his 
factors to the common factor space of the side correlations of the 
correlation matrix. 

Table 2 gives final factor matrices for a set of body measurements 
(Thurstone, 1946). The correlation matrix had three latent roots 
above unity. However, the fourth latent root was close to unity so 
four factors were extracted. The factor matrices seem to be identify- 
ing the same simple structure. For practical purposes the simple 
structures in this case are virtually identical. The factor matrices are 
based on the use of D,, obtained from F, (19), as an approximation 
to D;.5 


Comments, Conclusions and Summary 


The multiple factor analytic factor model represents intuitive 
ideal conceptions of common, unique, specific and error vectors. 
Vectors exist in the sample corresponding to these concepts. The 
concept of specific vectors is of peculiar importance for factor anal- 
ysis conceived as a tool for exploratory purposes, as a device for 
aiding an investigator to define and re-define a domain. Specific 
variance represents the acknowledgment and the assertion that 
part of what is unpredictable is real and the refusal to take refuge 
in the concept of what is in the nature of the case an indefinable 
population of factors. The specific vectors constitute a definition 
of the unrepresentativeness of tests with respect to populations of 
factors. When a battery of tests is representative of the population 
of factors, specific variance does not exist. However, a battery of 


oes 
5 Computer program UCSL 505 for obtaining D"D“RD,“D- is in the files 


of th i i i i 
fares ion Ga Program Library of the Computation Center of The 
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tests with zero specific variance may indicate only that some fac- 
tors in the domain are not represented at all in the battery of 
tests. Zero specific variance leads logically to the use of squared 
multiple correlations as communalities. 

Specific vectors were defined as the n orthogonal vectors obtained 
from iterating upon the vector sums of corresponding unit test 
vectors and unit unique vectors, These unit orthogonal vectors have 
the salient features of theoretically defined specific vectors: They 

1. are orthogonal, 

2. have high correlations with the corresponding test and unique 

vectors, 

3. have low correlations with the non-corresponding test and 

unique vectors, 

4. are identical with the unique vectors when the unique vectors 

are orthogonal. 

The squared correlation of the i-th unique vector with the i-th 
specific vector is the proportion of the variance of the unique vector 
accounted for by the specific vector. Multiplying this squared cor- 
relation by the squared correlation of the unique vector with the 
i-th test vector weights the maximum uniqueness value by the pro- 
portion of the unique variance accounted for by factors fraction- 
ally and poorly represented in the i-th test. The weight matrix Dy 
can then be applied to the regression equation for predicting one 
test from the remaining test to obtain 


F=C+D/D°F", 
The weighted regression equation then states that the error is D;’D* 
pathy than D*. As a matrix of uniqueness values the elements of 
D; D have the following properties not shared by the Guttman- 
Harris uniqueness values: 
1. they are always positive, 
2. they are always less than the elements of D? given the re- 


strictions on the correlations specified earlier and they prob- 
ably are always greater than “true” values, 


Meting pa correlation matrix by D;`D™, analogous to 
communalities, leads to the formulation of th 
the row vectors of oc. 


D,DE®’? — p-, 
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E(P? — P~”)F’ is congruent to diag. (I,, —I,-,) with index 
integer p. The index integer p is the same as Guttman’s strongest 
lower bound for the number of factors in the population when 
D™“RD* is analyzed instead of D; `D ”RD; >D. Here, however, 
the index integer may vary with the uniqueness values; its function 
is to indicate the maximum number of sample factors to be obtained 
from a given matrix of uniqueness values, weighted or unweighted. 
The index integer corresponding to D,*D? will be equal to or greater 
than the index integer corresponding to D’. However, the use of 
D,’D* will generally result in fewer final factors than the use of D° 
and should result in a simple structure better than or as good as 
that obtained from the use of D’, provided a simple structure exists 
in the data. Comparison of values of (P? — P™>) and (P — I)” 
shows that (P; — 1)” > P; — P,;-”. Furthermore, 
D; DE(P — I)? 

is a matrix for the reduced correlation matrix with communalities 
in the diagonals whereas D;DE(P’*” — P~”) is only a matrix 
of predicted scores. 


Since, the degrees of freedom available for testing p factors, 
p being the index integer before described, are 


df. = f° — (n + p)1/2 
where r = n — p, the inequality r? > n + p must hold if p factors 
can be tested for significance. For small correlation matrices it may 
be that 


<n +p. 
Thus, the maximum number of factors for a given D,’D* or D? 
might be too many factors to test for significance. 
The matrix 
EP(P pet IE’ aus EP” (P? afi Po) BY 
should have small inner products as elements but will have large 
inner products when the latent roots P; are in the range between 
2 and 1. Therefore, it seems that the number of factors to be ex- 
tracted should not be more than the number of roots, P;, equal to 
or greater than 2. If attention is centered upon common factors de- 
termined solely by the side elements of the correlation matrix, the 
number of latent roots of the correlation matrix above unity must 
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constitute its maximum number of factors. If, upon rotation, one 
essentially null factor is found, factors of D;DE (P — I)™2 should 
be taken from the factor matrix, one at a time, until only one null 
factor appears. This procedure corresponds to Thurstone’s recom- 
mendation that one more factor than necessary be taken before 
rotation, with one factor to be discarded after rotation if uninter- 
pretable or null. The idea is that simple structure is the final arbiter 
of the number of usable factors. The procedure of taking factors 
described will result in cleaner simple structures than the practice 
of taking many more factors for rotation than are finally retained 
provided: a simple structure exists in the data, 

The process of obtaining final factors should result in factor 
structures and in a number of factors similar to those obtained 
from an analysis in the manner of Thurstone. 
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TENSION IN FRESHMEN AND SENIOR 
ENGINEERING STUDENTS! 


BEN H. ROMINE, JR. 
AND 
: W. SCOTT GEHMAN 
Duke University 


Accorprne to the Taylor-Spence (1952) hypothesis, anxiety is 
said to facilitate performance when the performance task is a sim- 
ple one and to inhibit performance when the performance task is 
complex. In accordance with this hypothesis, Katzenmeyer (1958) 
and Spielberger and Katzenmeyer (1959) have concluded that anx- 
iety tends to interfere with academic performance, and Sarason 
(1957) found significant negative correlations between test anxiety 


coping ability. 


Duke University Computing Laboratory. 
565 


and academic grade average. On the other hand, Sarason (1957) 
also found significant positive correlations between general anxiety 
and academic grade average, and Suinn (1964) found that anx- 
iety level as measured by the Taylor Manifest Anxiety Scale (Tay- 
lor, 1953) is less predictive of academic performance than anxiety- 


Tf a conclusion can be drawn from the previously cited studies, it 
is that much more research is needed before any definite statements 
can be made about anxiety and its effects on academic performance. 

Sanford (1962) has been concerned with anxiety in college stu- 
dents and has concluded that the root of anxiety among college 
students is found in a conflict between impulse, ¢onscience, and ego. 
According to Sanford’s (1962) hypothesis, this conflict typically 
manifests itself among college students in the authoritarian per- 


‘ape ID 
1 All computations for this research were made through the courtesy of the 
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sonality. Other investigators, some of which are: Webster, 1958; 
Freedman, 1961; Webster, Freedman, and Heist, 1962; and Leh- 
man, 1963, have concluded that freshman college students tend to 
evidence more authoritarian personality traits than do senior col- 
lege students. The authors interpret the results of these studies as 
suggesting that college freshmen would tend to report more anxiety 
or tension than college seniors. 

The authors feel that the term “anxiety” is not adaptable to the 
concepts dealt with in the present study. English and English 
(1958) offer six definitions of “anxiety.” Without exception, these 
definitions characterize anxiety by reference to such terms as 
“fear,” “fearfulness,” “unpleasantness,” or “avoidance.” Olds 
(1955) concluded that a state of tension may be characterized by 
either positive (approach) or negative (avoidance and/or with- 
drawal) responses. Therefore it seems feasible to allow for the 
possibility of responses other than “fear,” “unpleasantness,” and/ 
or “avoidance” responses to situations assumed to be tension- 
producing. 

English, et al. (1958) define “tension” as “a condition of the or- 
ganism marked by restless activity, by pressure to act and readi- 
ness to act (but with no necessary implication of directed action).” 
(English, et al., 1958, p. 546.) In the present context, the authors 
feel that the term “tension” has the advantage of including the 
possibility of both positive (approach) and negative (avoidance 
and/or withdrawal) responses. For this reason, the term “tension” 
is employed in this study in lieu of the term “anxiety.” 

The purpose of the study is to determine whether or not there are 
any differences in the degree and kind of tension as reported by 
groups of freshmen and senior college students. Tension is defined 
in operational terms as that personality variable measured by 


Endler, Hunt, and Rosenstein’s Stimulus-Response Inventory of 
Anxiousness (SRI). 


It is hypothesized that: 

1. The mean total tension score e 
will be higher than and 
from the mean total tensi 
jects. (Degree of tension.) 

2. On each of the 14 mode-of-response scales, 


arned by the freshmen subjects 
statistically significantly different 
on score earned by the senior sub- 


the freshmen sub- 


i 
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jects will earn a mean score that is higher than and statisti- 
cally significantly different from the mean score earned by the 
senior subjects. (Kind of tension.) 

8. On each of the 11 situational scales, the freshmen subjects 
will earn a mean score that is higher than and statistically 
significantly different from the mean score earned by the 
senior subjects. (Kind of tension.) 


Procedures 


Subjects. The subjects were 86 freshmen male engineering stu- 
dents and 34 senior male engineering students enrolled in the Duke 
University College of Engineering for the academic year of 1964- 
1965. The freshmen were in their second semester of residence and 
the seniors were in the second semester of their fourth year of resi- 
dence. The freshmen were distributed in age from 17-19; the 
seniors were distributed in age from 20-24. 

Method. The SRI (Endler, et al., 1962) was administered in ac- 
cordance with standardized instructions to both freshmen and 
seniors the week following their return from spring vacation, The 
inventory was introduced to the subjects as a means of studying 
people’s reactions to and attitudes toward various types of situa- 
tions. The subjects were assured that their responses would be held 
in strict confidence. 

The freshmen subjects completed the inventory in four groups. 
One of the authors administered the inventory to all freshmen 
groups. The senior subjects completed the inventory in two groups. 
A professor of engineering administered the inventory to 14 seniors 
who were in his class. One of the authors administered the inven- 
tory to the remaining 20 seniors. As an incentive to and a reward 
for participation, the researchers offered all subjects an interpreta- 
tion of their scores as compared with scores earned by all partici- 
pants. 

Analysis of the data. In order to determine the degree of tension, 
the mean score for both freshmen and seniors, an F test of homo- 
geneity, and a ¢ test of the significance of the difference between 
means were computed for all responses to the inventory. The mean 
score represented a total tension score. 

In order to investigate the kinds of tension reported by fresh- 
men and seniors, mean scores earned by both groups and ¢ tests of 
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the significance of the difference between means were computed for 
each of the 14 mode-of-response scales and for each of the 11 situa- 
tional scales. 

Certain of the mode-of-response and situational scale scores 
were grouped into three mode-of-response factors and three situa- 
tional factors following the factor analysis of Endler, et al. (1962). 
Mean scores earned by both groups and ż tests of significance of the 
difference between means were computed for each of the factors. 

The situational scales which, in the opinion of the authors, rep- 
resented activities commonly associated with the college experience 
were combined, i.e., “Going into a psychological experiment,” “Go- 
ing to meet a new date,” “Getting up to give a speech before a 
large group,” “Going to a counseling bureau to seek help in solving 
a personal problem,” “Going into an interview for a very important 
job,” and “Entering a final examination in an important course.” 
The mean scores earned by both groups and a t test of significance 
of the differences between means were computed for this artifactual 
variable. The situational scales which, in the opinion of the au- 
thors, represented activities not commonly associated with the col- 
lege experience were combined, i.e., “Just starting off on a long 
automobile trip,” “Crawling along a ledge on a high mountain 
side,” “Starting out in a sail boat onto a rough sea,” “Entering a 
competitive contest before spectators,” and “Alone in the woods at 
night.” The mean scores earned by both groups and a ¢ test of the 
significance of the difference between means were computed for 
this artifactual variable. 

All ¢ ratios reported in this study were evaluated for level of 
vonfidence by means of a table of critical values of t, using the .05 
and .01 levels of significance for a one-tailed test. 


Results and Discussion 


Table 1 presents the mean scores and the t values of scores earned 
by freshmen and senior engineering students on the three mode- 


of-response factors, the 14 mode-of-response scales, and the total 
tension score. 


It can be seen from the data 


nibe presented in Tabl t 
of the significance of the differ ae 


ence between 
total tension score revealed ee ene 


no statistically significant difference 
between the mean total tension scores earned by the freshmen and 
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TABLE 1 


Mean Scores and t Values of Scores Earned by Freshmen and Senior Engineering 
Students on the Three Mode-of-Response Factors, the Fourteen 
Mode-of-Response Scales and the Total Tension Score 


Mode-of-Response Factor 


Fr. X Sr. X t 
Mode-of-Response Scale (N = 86) (N = 34) 
Exhilaration, enjoyment, approach 2.9197 2.9599 0.4167 
Feel exhilarated and thrilled 3.0941 3.1150 0.1918 
Enjoy the challenge 2.7082 2.7995 0.8270 
Seek experiences like this 2.9567 2.9652 0.0732 
Distress, disruption, avoidance 2.2336 2.1807 0.5879 
Heart beats faster 2.9894 2.9438 0.3616 
Get an uneasy feeling 2.7875 2.6497 1.0732 
Emotions disrupt actions 1.7611 1.7727 0.1004 
Want to avoid situation 2.3055 2.2487 0.5241 
Become immobilized 1.3245 1.2888 0.4426 
Autonomic reactions 1.6600 1.5134 1.8876* 
Need to urinate frequently 1.5359 1.4652 0.5810 
Experience nausea 1.2495 1.1390 1.4511 
Get full feeling in stomach 1.5159 1.4064 0.8816 
Mouth gets dry 1.9873 1.7086  1.9060* 
Perspire 2.4852 2.1979 1.9675* 
Have loose bowels 1.1860 1.1631 0.2791 
Total Tension Score 2.1450 2.0720 1.1121 


* Significant at or beyond the .05 level of confidence. 


senior samples. Therefore, the hypothesis that the mean total ten- 
sion score earned by freshmen subjects will be higher than and 
statistically different from the mean total tension score earned by 
the senior subjects was not supported. The F ratio resulting from a 
comparison of the total tension scores was 1.2369 with one degree 
of freedom. 

The é test of the significance of the difference between means 
computed for the various mode-of-response scales revealed that 
freshmen earned mean scores that were significantly higher than 
the mean scores earned by seniors on the “perspire” and “mouth 
gets dry” scales. Therefore, the hypothesis that on each of the 14 
mode-of-response scales the freshmen subjects will earn a mean 
score that is higher than and statistically significantly different 
from the mean score earned by the senior subjects is partially sup- 
ported. 

The ¢ test computed for the various mode-of-response factors re- 
vealed that freshmen earned a mean score that was statistically 


570 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


significantly higher than the mean score earned by seniors on the 
“autonomic reactions” factor scale. 

Table 2 presents the mean scores and t values of scores earned by 
freshmen and senior engineering students on the three situational 
factors and the 11 situational scales. 


TABLE 2 


Mean Scores and t Values of Scores Earned by Freshmen and Senior 
Engineering Students on the Three Situational Factors 


and the Eleven Situational Scales 
Situational Factors 
Fr. X Sr. £ 
Situational Scales (N = 86) (N = 34) t 
Ambiguous situations 1.6836 1.7742 1.6194 
Auto trip A 1.5914 1.5840 0.1224 
Psychological experiment 1.7757 1.9643 2.5433** 
Inanimate threats 2.0628 2.0420 0.2351 
Ledge on high mountain side 2.5565 2.4727 0.6362 
Sail boat on rough sea 1.7135 1.7290 0.1498 
Alone in woods at night 1.9186 1.9244 0.0583 
Tnterpersonal-status-threatened 2.3211 2.1674 1.9639* 
New date 1.9493 1.7668 2.1963* 
Speech before large group 2.7101 2.4916 1.8077* 
Counseling bureau 2.3331 2.2164 1.1929 
Competitive contest 2.1055 2.0924 0.1219 
Interview 2.2558 1.9076 3.5277** 
Final exam 2.5731 2.5294 0.3714 


* Significant at or beyond the .05 level of confidence, 
** Significant at or beyond the .01 level of confidence, 


It can be seen from the data presented in Table 2 that freshmen 
earned scores that were significantly higher than the scores earned 
by seniors on the following scales: “Going to meet a new date,” 
“Getting up to give a speech before a large group,” and “Going into 
an interview for an important job.” Therefore, the hypothesis that 
on each of the 11 situational scales the freshmen subjects will earn 
a mean score that is higher than and statistically significantly dif- 
ferent from the mean score earned by the senior subjects was par- 
tially supported. 
i On the “Going into a psychological experiment” scale, the sen- 
tors earned a mean score which was Significantly higher than the 


it is the opinion of the authors that 
least one course in psychology, they 
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might be more sophisticated with regard to experiments involving 
deception, noxious stimuli, and the like. 

The ¢ test of the significance of the difference between mean 
scores for the various situational factors revealed that freshmen 
earned a mean score that was significantly higher than the mean 
score earned by seniors on the “interpersonal status threatened 
factor” scale. 

Upon further study of the data, the authors recognized the 
emergence of several patterns of responses to both the mode-of-re- 
sponse and the situational scales which seemed worthy of further 
analysis. For the two groups, six scales yielded absolute mean dif- 
ferences which, according to the null hypothesis, would be expected 
to occur by chance alone on only one out of four administrations of 
the instrument (.25 level of confidence). With 118 degrees of free- 
dom, a ¢ value of 0.6766 is required to establish the .25 level of 
confidence. The authors are aware of the dangers inherent in em- 
ploying levels of confidence less than .05 in identifying trends and 
tendencies. Therefore, in the discussion to follow, the reader must 
bear in mind the tentativeness of the suggestions and avoid reach- 
ing conclusions in the absence of statistical support. 

As can be seen from the data presented in Table 1, of the mode- 
of-response scales which yielded ¢ values of 0.6766 (.25 level of 
confidence) or greater, but did not meet an acceptable level of sig- 
nificance adopted for this study, the freshmen earned higher mean 
scores than the seniors on the following: “Get an uneasy feeling,” 
“Get full feeling in stomach,” and “Experience nausea.” The “Get 
an uneasy feeling” scale was included in the “distress, disruption, 
and avoidance” factor in the original factor analysis , (Endler, 
et al., 1962). Endler, et al. (1962) state that this scale “point(s) to 
autonomic reactions.” (Endler, et al., 1962, p. 27.) The “Get full 
feeling in stomach” and “Experience nausea” scales were included 
in the “autonomic reaction” factor in the original factor analysis 
(Endler, et al., 1962). With regard to the six scales included in the 
“autonomic reactions” factor, it can be noted that in every case the 
freshmen earned higher mean scores than the seniors, although not 
always statistically significantly higher. 

As noted previously, the freshmen earned a mean score that was 
statistically significantly higher than the seniors on the “autonomic 
reactions factor” scale. It is the authors’ opinion that this finding 
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suggests a tendency for freshmen college students to respond to 
tension-producing situations of the nature of those presented in the — 
S-R Inventory (Endler, et al., 1962) with a greater degree of 
autonomic reactions than do senior college students. From the data 
at hand, the implication of this finding is a matter for conjecture, 
One possible interpretation is that college freshmen, having met 
and dealt with fewer tension-producing situations than college 

» seniors, have not developed the tension-reducing mechanisms which 
are available to the more experienced seniors. Stated in dynamic 
terms, Sanford (1962) has hypothesized that in the case of the 
college freshmen 


. .. the controls developed for the purpose of inhibiting impulses — 
are still unseasoned and uncertain; they are likely to operate in — 
a rigid manner, that is, to be rather overdone, as if the danger of 
their giving way altogether were still very real. (Sanford, 1962, 
p. 260.) 


Tf the Sanford (1962) hypothesis is accepted, it may be said that 
the college freshman, whose impulse-control mechanisms are “still 


unseasoned and uncertain” (Sanford, 1962, p. 260), would tend to. 


report more autonomic reactions to tension-producing situations — 
than would college seniors who have had more opportunities to 
learn and employ more highly-developed control mechanisms. 

As can be seen from the data presented in Table 1 , of the mode- 
of-response scales which yielded ¢ values of 0.6766 (.25 level of 
confidence) or greater, but did not meet acceptable levels of con- 
fidence adopted for this study, the seniors earned a mean score 
which was higher than that earned by freshmen on the “Enjoy the — 
challenge” scale which was included in the “exhilaration, enjoy- 
ment, and approach” factor in the original factor analysis (Endler, 
et al., 1962). It is noteworthy that on all three of the scales in- 


cluded in the “exhilaration, enjoyment, and approach” factor, the — 


Seniors earned higher mean scores than the freshmen although none 
of the differences were statistically significant. 

; As can be seen from the data presented in Table 2, of the situa- 
tional scales which yielded t values of 0.6766 (.25 level of con- 
fidence) or greater, but did not meet the level of confidence adopted 
for this study, the freshmen earned a mean score which was higher 
than that earned by the seniors on the “Going to a counseling 
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bureau to seek help in solving a personal problem” scale. This 
scale was included in the “interpersonal status threatened” factor 
of the original factor analysis (Endler, et al., 1962). With regard 
to the six scales included in the “interpersonal status threatened” 
factor, it can be noted that in every case the freshmen earned a 
higher mean score than the seniors, although not always statisti- 
cally significantly higher. As has been previously noted, the fresh- 
men earned a mean score that was significantly higher than the 
seniors on the “interpersonal status threatened” scale. 

The implication of this finding is open for conjecture. One pos- 
sible explanation is that college freshmen, having had fewer op- 
portunities (relative to college seniors) to meet and deal with situ- 
ations in which their interpersonal status is threatened, would tend 
to report more tension than seniors. Vogel, Raymond, and Lazarus 
(1963) concluded that subjects with past successes in meeting 
achievement and affiliative needs experienced less stress (tension) 
in tension-producing situations than subjects with a history of fail- 
ure in these areas. 

In addition to the mode-of-response and situational scales which 
yielded ¢ values of 0.6766 (.25 level of confidence) or greater, but 
did not meet acceptable levels of confidence, a comparison of the 
total tension mean score earned by the freshmen and senior groups 
also yielded a ¢ value greater than 0.6766 (.25 level of confidence). 
The freshmen tended to earn a higher total tension mean score than 
did the seniors, 

It is hypothesized by the authors that had the S-R Inventory 
(Endler, et al., 1962) been administered earlier in the academic 
year, more significant differences between mean scores earned by 
freshmen and seniors would have been revealed. The basis for men- 
tioning this possibility lies partially in the fact that the trends 
identified in the present study seem to suggest some significant dif- 
ferences between mean scores earned by freshmen and seniors on 
certain of the mode-of-response and situational scales and factors. 
Much more important than this, it would seem, is the report by 
Freedman (1961) that “studies of juniors, sophomores, and fresh- 
men in their second semester indicate that . . . senior-freshmen 
changes are not linear from year to year but tend to take place 
quite early.” (Freedman, 1961, p. 22.) 

After analyzing the data and observing the tendency for fresh- 
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men to earn higher mean scores than seniors on those scales which 
might intuitively be considered to involve activities commonly as- 
sociated with the college experience, the authors combined all 
such scale scores and computed a t test of significance of the dif- 
ference between the means of the freshmen and senior groups. 

Table 3 presents the mean scores and £ values of scores earned 
by freshmen and senior engineering students on the combined situ- 
ational scales which involve activities not commonly associated 
with the college experience. 


TABLE 3 


Mean Scores and t Values of Scores Earned by Freshmen and Senior Engineering 
Students on the Combined Situational Scales Which I; nvolve Activities 
Commonly Associated with the College Experience, and the Combined 

Situational Scales Which Involve Activities Not Commonly 
Associated with the College Experience 


Fr. X Sr. X 
Combined Scale (86) (34) t 
College experience 2.2662 2.1460 1.6935* 
Non-College experiences 1.9771 1.9605 0.2282 


* Significant at or beyond the .05 level of confidence, 


As can be seen from the data presented in Table 3, the ¢ test of 
significance of differences between means computed for the combined 
situational scales taken to represent situations which involve activi- 
ties commonly associated with the college experience, revealed that 
the freshmen earned a mean score which was higher than, and sta- 
tistically significantly different from, the mean score earned by 
seniors. The £ test of significance of the difference between means 
computed for the combined situational scales which involved activi- 
ties not commonly associated with the college experience, revealed 
no differences between the mean scores earned by freshmen and 
seniors, 

The authors interpret this observation as Suggesting that as col- 
lege students gain experience, the tension likely to be produced by 


situations such as those presented in the S-R Inventory (Endler, 
et al., 1962) tends to be diluted, 


Summary 


This study represents an attempt to determine whether or not 


there are any differences in the degree and kind of tension as re- 
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ported by groups of freshmen and senior engineering students. 

In operational terms, tension is defined as that personality vari- 
able measured by the Stimulus-Response Inventory of Anxiousness 
(SRI) as constructed by Endler, Hunt, and Rosenstein (Endler, 
et al., 1962). 

Eighty-six freshmen and 34 senior male engineering students 
completed the S-R Inventory under standardized procedures. 

In order to determine the degree of total tension reported by 
freshmen and seniors, the mean score for both groups and a £ test 
of significance of differences between means were computed for all 
responses to the inventory. In order to investigate the kinds of 
tension reported by freshmen and seniors, mean scores earned by 
both groups and t tests of significance were computed for each of 
the 11 situational scales and for each of the 14 mode-of-response 
scales. In addition to these scores, certain of the mode-of-response 
and situational scale scores were grouped into three mode-of-re- 
sponse and three situational factors following the factor analysis of 
Endler, et al. (1962). Mean scores earned by both groups and t 
tests of the significance of differences between means were computed 
for each of these factors. The situational scales which, in the 
opinion of the authors, represented activities commonly associated 
with the college experience were combined. The mean scores earned 
by both groups and a ¢ test of the significance of the difference be- 
tween means were computed for this artifactual variable. The situ- 
ational scales which, in the opinion of the authors, represented 
activities not commonly’ associated with the college experience 
were combined. The mean scores earned by both groups and a ¢ test 
of the significance of the difference between means were computed 
for this artifactual variable. 

An analysis of the data revealed no statistically significant dif- 
ference between freshmen and seniors with regard to the degree of 
total tension as defined by the mean scores earned on the 8-R In- 
ventory (Endler, et al., 1962). There were statistically significant 
differences between the mean scores for freshmen and seniors with 
regard to the kind of tension as measured by certain mode-of-re- 
sponse scales presented in the inventory. Furthermore, there were 
statistically significant differences between the mean scores for 
freshmen and seniors with regard to the types of situations which 


produced tension. 
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The authors recognize the S-R Inventory (Endler, et al., 1962) 
as a valuable research instrument. 
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IMPROVED HIERARCHICAL SYNDROME ANALYSIS 
OF DISCRETE AND CONTINUOUS DATA 


t 
LOUIS L. McQUITTY 
Michigan State University 


In the isolation of statistically defined types, Hierarchical Syn- 
drome Analysis has the advantage of being simple, brief, and ap- ; 
plicable to both discrete and continuous data. In its original de- . 
velopment, it does, however, have shortcomings; once Individual 
i has classified with Individual j then he cannot by himself classify 
with any individual other than j, and this is true even though j 
and k each have i most like them. In order to correct for this limi- 
tation, a version was developed which classifies 1 with each j and 
k in the above circumstance, but the latter version has the dis- 
advantage of being cumbersome and of yielding unnecessarily com- 
plicated classifications (McQuitty, 1960). 


An Improvement 


An improvement is realized by combining certain features of each 
(a) Hierarchical Analysis by Reciprocal Pairs (McQuitty, 1964, 
1966), and (b) Hierarchial Syndrome Analysis (McQuitty, 
1960); the theory of types from (a) is applied to a new version of 
the approaches of (b). More specifically, the problem between t 
and j, as outlined above, is solved by defining types in such a fash- 
ion that the problem disappears; the analysis then proceeds with- 
out internal inconsistencies. 

. An hierarchical classification is a successive categorization of 
imperfect types from one level into more exacting representation of 
types at the next higher level. Each successive level contains fewer 
categories than the previous one. Each category reflects only one 
imperfect type; it is defined by the characteristics of the individ- 
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uals in the category. Single individuals represent imperfect types 
at the first level of classification. 

An hierarchical type is represented by a pair of imperfect types 
from the next lower level of classification; each of these imperfect 
types is more like the other one of them than it is like any other 
imperfect type at the same stage of classification. 

Hierarchical types are intermediate between those represented 
in reality by single individuals and the pure types of theory, which 
are approached but never quite realized, as individuals are classi- 
fied in higher and higher levels. 

By the above definition, ij constitutes a type if, and only if, the 
association between i and j is larger than that between i and k, 
and on the other hand, ik constitutes a type if, and only if, the as- 
sociation is reversed; 7 and k is larger than i and j. If the associ- 
ations are tied, neither ij nor ik constitute a type. This is the prin- 
ciple of classification by reciprocal pairs (McQuitty, 1964, 1966). 


For the purpose of further clarification assume: 
a. tis highest in Column j 
b. iis highest in Column k 
c. j is highest in Column i; k is not highest in Column 7; 7 and 
j are reciprocal 
d. ty > Tx 


Then by the above definition of types, i and j would join to form 
an hierarchical type which is a better representative of a pure type 
than is either 7 or j separately. In the analysis, the two poor rep- 
resentatives, i and j, of a single hierarchical type, ij, would be re- 
placed by tj. It is now irrelevant that 7 is highest in Column k, and 
the problem has been eliminated. 

The above discussion illustrates a very important characteristic 
of the method, The method is designed to improve and purify the 
data as it proceeds; each successive category is intended to become 
a better representative of a type as it exists in theory. 

Ih summary, the method searches for evidence of types. A re- 
ciprocal pair (i highest with j and j highest with i) is accepted as 
such evidence. The members of a pair are combined and the two 
Jointly are accepted as a better representative of a type than is 
either separately. 


The above definition of an hierarchical type enables us to solve 
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another problem central to our classification procedures, It justifies 
the classification assumption (McQuitty, 1960): an hierarchical 
type has as much association with another hierarchical type as 
the smallest association represented by all pairs of persons involved 
in either one or both of the two types. This assumption maintains 
that the upper limit in the size of the association (as represented 
by the lowest pair) is, in fact, also the lower limit. 

In order to perform perfectly, the assumption need not be per- 
fectly valid; it need only be relatively valid. It must preclude 
non-typal categories from competing successfully with typal cate- 
gories, and non-typal categories are characterized by extremes in 
low pairs, while typal categories are in general just the opposite. 


ka 


» Method 
¥ The method is illustrated by application to the data of Matrix 1, 
| Table 1, coefficients of correlations between people for the second 
through the sixth person of a matrix from Stephenson (1953, p. 
| 169). 
TABLE 1 
An Analysis of Coefficients of Correlation between 3i 
BCDEF B CD EF B CD EF BCD EF 


B 60 49 24 43 B 49 24 43 B 49 24 BCD 24 
C 60 62 46 51 CD 49 40 51 CD 49 40 EF 24 
D 4962 40 57 E 24 40 56 EF 24 40 
E 2446 40 56 F 43 
| F 43 51 57 56 f 
P Matrix 1 Matrix 2 Matrix 3 Matrix 4 


Note.—Data of Matrix from Stephenson (1953, p. 169); the entries have been rounded from three to two 
place numbers, The decimal points have been omitted. 


The first step is to underline the highest entry of each column of 
Matrix 1, 60 for Column B, 62 for C, 62 for D, 56 for E, and 56 for 
F. Then, by selecting the highest underlined entry, one selects the 
highest entry in the matrix; it is 62 and appears in both Columns 

| Cand D. 
| Matrix 2 is formed from Matrix 1 by collapsing Columns C and 
| D of Matrix 1 into a single Column CD in Matrix 2. In determin- 
ing the entry for the top cell of Column CD (Row B—Column CD), 
A one looks to two cells of Matrix 1: (a) Row B—Column C with an 
entry of 60, and (b) Row B—Column D with an entry of 49. The 
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smaller of these two entries, 49, is the entry of Row B—Column 


CD, Matrix 2. In case of a tie between these two values, the tied 


value is used. 

The other entries of Column CD, Matrix 2, are determined in a 
similar fashion. The entries of Row CD are the same as those of 
Column CD. The other entries of Matrix 2 are realized by copying 
the corresponding entries from Matrix 1. 

Each subsequent matrix is formed from its predecessor in the 
same fashion as Matrix 2 was formed from Matrix 1. For example, 
the highest entry in Matrix 3 is 49 in Columns B and CD. In form- 
ing Matrix 4, these two columns are collapsed into Column BCD. 
The entry for Row EF—Column BCD of Matrix 4 is the smaller 
of the following two entries of Matrix 3: (a) Row EF—Column B 
which is 24, and (b) Row EF—Column CD which is 40. Conse- 
quently, 24 is the entry for Row EF—Column BCD (and for Row 
BCD—Column EF) of Matrix 4. 


Results 


Matrices 2 through 4 reflect the classification. It is summarized 
in Figure 1 which shows that Persons C and D first joined one 
another with an index of 62, and were followed in order by E and F 
with an index of 56, B and CD with an index of 49, and finally 
BCD and EF with an index of 24. 


A Critique of the Method 


In the present approach, we eliminated but one reciprocal pair at 
a time (the highest one) in forming a new matrix. We could have 
eliminated all of them each time we formed a new matrix, and this 
is the way in which the replacement version, as originally de- 
seribed, does in fact proceed (McQuitty, 1960). The highest-pair 
version, as here described is, however, less complicated. 

A word of precaution is essential. The highest pair of a matrix is 
always reciprocal, and if an analysis yields only one reciprocal 
Pair in each of its successive matrices, it is well to realize that this 
could occur from chance alone, but at the same time 
to be due to chance alone. One nee i 
the meaningfulness of the categori 
other than chance. 


does not need 
ds additional evidence, such as 
es, to support an interpretation 
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Coefficients of Correlations for the Lowest 
Pair of Each Category 
ou 
oO 


100 B Cc D E F 


Figure 1. Classification of people from a matrix of intercorrelations between 
pairs of them. 


Summary 


This paper develops a new version of Hierarchical Syndrome 
Analysis, called the highest-pair version; this version is simpler 
both to describe and apply than either of the two earlier versions, 
the replacement version and the self-checking version. 

In developing the new version, this paper shows how both the 
new version and the replacement version can be interpreted to im- 
prove and purify the data as they proceed; evidence of types is 
sought and then the analysis is intended to improve on the descrip- 
tions of the types as the analysis proceeds. 

All three versions are applicable to both discrete and continu- 


ous data. 
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THE EFFECT OF VARIATIONS IN THE CUE R 
MATRIX UPON THE OBTAINED POLICY EQUATION 
OF JUDGES 


ARTHUR L. DUDYCHA anp JAMES C. NAYLOR 
The Ohio State University 


In several earlier articles the multiple regression model has been 
demonstrated to be an extremely useful methodology for describing 
the strategies or policies of judges with respect to their responses to 
a set of complex stimuli (Naylor and Wherry, 1965; Wherry and 
Naylor, 1966). In such studies the usual procedure is to relate each 
of the k characteristics of the stimuli to the responses made by the 
judge by means of a least squares prediction equation 


Y; = bi*X1, + b Xn + +++ + Xu 


where 


Y,’ = predicted response of a judge when presented 

with stimulus z 

(b,*, bo*, «++, b,*) = least square regression weights for the judge in 
standard score form for each of the k stimulus 
dimensions. They are obtained by computing 
a multiple regression equation (usually over 
all n stimuli) using the different stimulus cues 
as predictors and the judge’s response as the 
criterion. 

(Xin Xo;, ++, Xu) = values, in standard score form, for each of the 
different cue dimensions for stimulus 7. 

Now any set of n stimulus objects presented to a judge can itself 
be described in terms of an R matrix which is formed by intercor- 
relating the different cue values over all stimuli, thus 
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Naylor and Wherry (1964) have raised the question of the im- 
portance of this R matrix as a determiner of a judge’s equation or 
policy. Will one obtain similar policies using different sets of 
stimuli which in turn have different R matrices? The question would 
certainly seem to be a basic one, since at best the n stimuli used in 
an experiment represent a truly random sample of stimuli drawn 
from a population size N. However, in many cases, due to various 
practical expediencies, our experimental stimuli may not be truly 
“representative” of the population to which they belong, i.e., 
Rexe. will not be representative of Rerun. For example, in some cir- 
cumstances the R matrix is deliberately made an identity matrix in 
order to allow Æ to assess the “independent contribution” of each 
cue dimension. While one may applaud the objective, it raises the 
question as to whether the resulting policies are truly those that one 
would obtain using stimuli having an R matrix more truly repre- 
sentative of Rraun.t 

The purpose of the present research study was to determine what 
influence distorting the R matrix from Rrrvn would have upon the 
rater policy equations obtained using the multiple regression model. 


Method 


Experimental Stimuli 


The stimuli used in the study were job “profiles”—that is, each 
stimulus represented a particular job which had been given a score 
on each of a number of different job dimensions. Subjects were 
required to examine each job and give it a rating in terms of its 
overall “Job Desirability.” The job characteristics which were used 
were those obtained in an orthogonal re-rotation of the Baehr 


> Of course there is always the possibili that R is i identi 
matrix. However, the authors doubt that as will re aie sae, Tr 
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(1954) and Ash (1954) factor analytic studies of the SRA Em- 
ployee Inventory (presently known as the SRA Attitude Survey) 
by Wherry (1954). The R matrices from the Baehr and Ash studies 
(two from Baehr’s study and one from Ash’s) were combined into 
a single R matrix by computing the weighted averages for each in- 
tercorrelation of the fourteen subtests of the SRA Attitude Survey. 
The resulting matrix was assumed to describe the “real” world in- 
terrelationships of the 14 subtests (job traits) of the SRA Attitude 
Survey and was called Rrror. In a similar manner the three respec- 
tive orthogonally rotated factor structures from Wherry (1954) 
were combined, yielding the theoretical F matrix associated with 
Rrror. This final Frros matrix contained six factors—one general 
and five subgeneral—plus 14 specifics as being appropriate to de- 
scribe the underlying dimensions necessary to explain the job traits. 
The six factors were (G) General Bias or Attitude, (A) Working 
Conditions and Requirements (status), (B) Financial Reward, (C) 
Immediate Supervision, (D) Effective Management and Adminis- 
tration, and (E) Security and Communication (see Table 1). 


Distorted R Matrices 


Two related factor structures with their respective correlation 
matrices were also created, holding the reliabilities (r4) constant. 
These two factor structures and their correlation matrices were dis- 
tortions of Freon and Ryrgun. The first distortion (Dz) was obtained 
simply by modifying Frrun into a completely orthogonal or inde- 
pendent F matrix containing the square root of the Frrun reliabili- 
ties (Vra). in the diagonal, while the second distortion (Dg) was 
an F matrix obtained by systematically collapsing all of the spe- 
cifics of the Freon matrix into the appropriate six factors of this 
factor structure. Thus, three distinct theoretical F matrices and 
their associated theoretical R matrices were defined to represent 
three theoretical sets of stimuli (see Tables 2-4). 


Profile (Stimuli) Generation 


Three sets of two hundred job profiles were produced (one set 
from each of the previously created F matrices) using the Ohio 
State Correlated Score Generation Method, (Wherry, Naylor, 
Wherry and Fallis, 1965). Each profile under each condition 
presented a score for that “job” on each of the fourteen traits. The 
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TABLE 2 
i (Condition T) 
Trait Theoretical Intercorrelations (Above Diagonal), Reliabilities (Diagonal) 
and Trait Empirical Intercorrelations (Below Diagonal) 


Trait Number* 
1 2 3 4 5 6 
1 100 31 30 42 41 43 
2 35 100 23 28 42 41 
3 24 19 100 35 34 34 
4 37 26 35 100 52 45 
5 40 42 25 54 100 49 
6 36 26 26 31 46 100 


Note.—In this and the two subsequent tables, the figures are given correct to two decimal 
places and the decimal points are omitted. 


a The trait numbers in this and the two subsequent tables refer to the following trait names: 


1, Working Conditions 

2. Pay 

3. Fellow Employees 

4, Supervisor-Employee Relations 

5, Security 

6. Opportunity for Growth and Advancement 


TABLE 3 
(Condition D,) 
Trait Theoretical Intercorrelations (Above Diagonal), Reliabilities (Diagonal) 
and Trait Empirical Intercorrelations (Below Diagonal) 


Trait Number 
YO Le SSS See ee ee 
1 2 3 4 5 6 

de Se ee SS ee a 

1 100 00 00 00 00 00 

2 —09 100 00 00 00 00 

3 —04 07 100 00 00 00 

4 01 -01 —02 100 00 00 

5 —04 —02 03 —06 100 00 

6 06 04 —03 —03 03 100 
She 

TABLE 4 
(Condition Dy) 


Trait Theoretical Intercorrelations (Above Diagonal), Reliabilities (Diagonal) 
and Trait Empirical Intercorrelations (Below Diagonal) 


Trait Number 
fadoh iid SO Ae D E le 
1 2 3 4 5 6 

nn neve cS OE 

1 100 37 61 60 63 70 

2 39 100 32 27 43 46 

3 60 31 100 39 81 42 

4 59 26 47 100 65 50 

5 56 37 76 64 100 51 

6 64 42 34 44 45 100 
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scores ranged from 1 (very poor) to 9 (very good) with 4 through 
6 considered an average range. The two hundred scores for each 
trait were normally distributed, and the intercorrelations of profile 
trait scores differed only by sampling error from the appropriate 
theoretical “population” correlation matrix obtained directly from 
the underlying F matrix used to generate the profile scores. 

Thus, three different sets of experimental stimuli conditions were 
created: Condition T, regarded as the control condition, consisted of 
the profiles based on the Reeve matrix and was considered as being 
representative of the “real” world situation; Condition Dz, which 
contained profiles generated from the orthogonal F matrix yielding 
intercorrelations approximating zero; and Condition Dy, which con- 
sisted of profiles generated from the collapsed Frros matrix produc- 
ing relatively high intercorrelations, 

Because of testing time limitations it was necessary that the 
actual stimuli presented to each S only include six of the fourteen 
original job traits [(1) Working Conditions, (2) Pay, (3) Fellow 
Employees, (4) Supervisor-Employee Relations, (5) Security and 
(6) Opportunity for Growth and Advancement]. Tables 2, 3 and 4 
show the theoretical and empirical R matrices for these six job 
traits. The six job traits describing the profiles were selected on the 
basis of meaningfulness to the rater population relative to the 
criterion, “Job Desirability,” and as being the traits most likely to 
be relevant to the employee’s perception of the work environment. 


Subjects 


One hundred and fourteen male introductory psychology stu- 
dents at Ohio State University served as raters in this study who, 
in doing so, partly fulfilled their course requirements of experi- 
mental participation. They were divided into three equal groups of 
88 members each. The selection of raters for a Particular group 
was based strictly on their availability for testing rather than by 
any random assignments to the three respective groups. 


Procedure 


(one set of two hundred profiles for 
onto a 180 foot roll of typing paper 

a specially constructed device designed 
to feed the Paper continually into an Opaque projector. Preceding 
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each set of two hundred profiles were three examples. Hach indi- 
vidual profile within a condition set was successively numbered 
from 1 through 200, and was not identified by any specific job title. 
Through the use of a black template, on which the individual job 
traits were identified, individual job profiles were able to be dis- 
played sequentially from 1-200 to subjects in a partially darkened 
room. Each profile was displayed for approximately 12-15 seconds, 
the length of time being controlled by Æ. The job traits were listed 
in the same order for all raters. Figure 1 shows a sample profile 
displayed on the screen as viewed by the rater. 


$ 
gs 
<Y 
~ 
& a 
Ý g es 
A 3 Pa Ro 
S X ot RSS 
ie) C @ A Soe 
Pie eh 
RO HS ANS ES 
eT LY SF & FF 


pae Ee ae B T aS 


Figure 1. Sample “job” profile as viewed by the raters. 


Raters were tested in groups of ten to twenty in size depending 
upon their availability for testing, with 38 different raters assigned 
to each condition. Each rater was given a set of directions describ- 
ing the experiment and his task, a list of the six job traits with their 
descriptors, and an answer sheet with two hundred numbered 
blanks—one for the rater’s evaluative response to each profile. 
Each was instructed that each individual profile was representa- 
tive of a particular job. He was told to examine each profile and 
give it a score from 1 (very undesirable) to 9 (very desirable) in 
terms of the criterion, “Job Desirability.” 


Results 


- Regression or policy equations for each rater were computed us- 
ing the generated profile job trait scores as predictors (X,) and his 
response or profile evaluation relative to “Job Desirability” as the 
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criterion. These regression equations, which represent the policies 
or strategies used by the raters in evaluating the profiles, were then 
sequentially clustered within each condition using the JAN tech- 
nique of Bottenberg and Christal (1961) in an attempt to identify 
groups of raters holding heterogeneous subpolicies. This was fol- 
lowed by an analysis of variance using the validity coefficients of 
the predictor traits in order to examine the results for possible be- 
tween-cluster differences, between-condition differences, and for 
cluster-by-condition differences. Finally, a principal axis factor 
analysis followed by a varimax (Kaiser) rotation was performed 
on an R matrix (114 X 114) based upon the intercorrelations be- 
tween the validity vector of each rater with those of all other 
taters across all three conditions. The factor analysis of these rater 
validity vectors (PROF Technique—Wherry and Naylor, 1966) 
was performed to compare the obtained rater factor strategies in 
the three conditions. 


Regression Equations (Policy Predictability) 


The results of the regression analyses performed upon the profile 
evaluations of each judge are summarized in Table 5. The table 
gives the squared multiple correlation coefficient (Rê) obtained for 
each rater. R; is an expression of the consistency of a rater’s judg- 
ment across all two hundred profiles. The R2 values show that while 
in general all raters exhibited a high level of intrajudge consistency, 
there were certainly substantial differences between raters within 
the respective conditions. For example, the R? values for Condition 
T ranged from a high of .905 (Rater 4) to a low of .522 (Rater 23); 
from .923 (Rater 21) to .490 (Rater 36) for Condition Dy; and 
from .940 (Rater 14) to 522 (Rater 8) for Condition Dg. How- 
ever, there is a very distinct and systematic difference between the 
relative magnitudes of the R, values for each separate condition, 
with raters in Condition Dy being most systematic, raters in Con- 
dition Dy, being least systematic, while raters in Condition T fall 
between the previous two conditions. 

The Taw score regression weights (b;) given to each of the six 
predictor variables (job traits) by a rater and the associated in- 
tereept were also obtained. The data presented in Table 6 sum- 
marize the results obtained by computing the average regression 
weight for each of the six Predictor variables across each of the 
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TABLE 5 


Squared Multiple Correlation Coefficients (R.2) for the Thirty-eight 
Raters in Each Experimental Condition 


(Rater Consistency) 
STE SST eS MEP TA EG ROE E aN A N EY 
Condition 
Rater T, De Da 
eee EEN S ARANDA W E E E ERSE 
1 859 670 915 
2 874 794 875 
3 774 705 912 
4 905 654 914 
5 827 736 912 
6 807 774 910 
* 7 877 TAT 920 
8 879 728 522 
9 847 701 913 
10 891 784 911 
11 701 666 827 
12 872 758 829 
13 804 785 890 
14 833 816 940 
15 872 679 923 
16 891 658 894 
17 858 515 907 
18 765 807 900 
19 858 761 909 
20 831 778 896 
21 846 923 900 
22 852 848 916 
23 522 789 869 
24 836 725 887 
25 875 681 868 
26 862 797 834 
27 800 759 889 
28 868 707 910 
29 857 870 840 
30 847 778 912 
31 883 774 859 
32 848 802 901 
33 823 740 879 
34 897 775 896 
35 896 687 904 
36 853 490 896 
37 882 774 915 
38 899 815 907 


three conditions. Note that the rank order and the relative magni- 
tudes of the average regression weights are very similar under 
each of the three conditions. The job traits, Pay and Opportunity 
for Growth and Advancement, were quite consistently weighted 
high by most raters within each condition, while Fellow Employ- 
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TABLE 6 
Average Raw Score Regression Weights (5;’s) Associated With Each 
of the Profile Job Traits for Each Condition 
a e 


Conditions 

Job Traits T D Dy Total 
An ei E abioi oad. eee a 
Working Conditions 174 (3) 154 (4) 120 (5) 149 (4) 
ae 351 (1) 461 (1) 859 (1) 390 (1) 
Fellow Employees 112 (6) 132 (5) 136 (4) 127 (5) 
Supervisor-Employee Relations 115 (5) 112 (6) 100 (6) 109 (6) 
Security 159 (4) 199 (3) 143 (3) 167 (3) 

rtunity for Growth and 

his ui 331 (2) 361 (2) 335 (2) 342 (2) 


Note—In this table the figures are given correct to three decimal places with the decimal points 
omitted. P 
3 The values in the parentheses represent the rank order of the job traits within that condition, 


ees, and Supervisor-Employee Relations consistently received low 
weightings by most raters across conditions. 


Policy Comparisons (Policy Comparability) 


In order to evaluate the raters’ policy equations obtained using 
the different underlying cue R matrices, several comparisons seemed 
important. First, would the same group policies emerge, that is, 
would the same relative weights be given to the predictor variables 
(stimulus cues) under each condition when averaged across all 
raters in that condition? Second, if there exist different subpolicies 
among the raters within a condition, will the same subpolicies 
emerge under each condition? The basic index of a rater’s policy 
adopted to examine the previous questions was that rater’s validity 
vector, since prior research by Naylor and Wherry (1964) had in- 
dicated the validity vector to be a more stable index than a vector 
of regression weights. 

First, the raters’ regression equations were sequentially clustered 
within each condition using JAN (Bottenberg and Christal, 1961). 
The JAN iterative clusteri g criteria minimized the differential com- 
posite squared multiple (AR) for the entire group, retaining op- 
timum predictive efficiency at each stage in the clustering process. 
Thus, AR? = Ra? — Ro’, where Ra? was defined as the com- 
posite group predictive efficiency at stage i in the clustering process 
and ,? as the composite group predictive efficiency at stage j(j = 
ite 1), the next successive clustering stage. The composite predic- 
tive efficiency at any stage 7 (Ra?) , was defined as 
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R2= SSregi(Ni) + SSrego(N2) + +++ + SSregu_i(Ni-i) 
FA SSrotats F SSrotats aR eR, rr SSrotats 
where k is the original number of regression equations to be grouped 
and 7 is the number of stages (Naylor and Wherry, 1964). 
Table 7 presents the computed values of R2 and AR for each of 


TABLE 7 
Composite Predictive Efficiency (Rè) at Each Stage of the Grouping 
Process for Each of the Three Conditions 


Condition T Condition Dy Condition Dy 
System Rè AR2* Rè AR? Rè ARA 
38 8456 0007 7597 0002 8883 0002 
37 8449 0017 7595 0026 8881 0013 
36 8432 0001 7569 0061 8868 0002 
35 8431 0065 7508 0002 8866 0021 
34 8366 0009 7506 0009 8845 0001 
33 8357 0003 7497 0013 8844 0001 
32 8354 0021 7484 0003 8843 0002 
31 8333 0004 7481 0047 8841 0009 
30 8329 0003 7434 0045 8832 0002 
29 8326 0012 7389 0017 8830 0002 
28 8314 0004 7372 0007 8828 0003 
27 8310 0006 7365 0081 8825 0004 
26 8304 0024 7284 0007 8821 0004 
25 8280 0006 7277 0031 8817 0005 
24 8274 0008 7246 0009 8812 0004 
23 8266 0007 7237 0031 8808 0014 
22 8259 0007 7206 0016 8794 0006 
21 8252 0007 7190 0015 8788 0005 
20 8245 0005 7175 0012 8783 0011 
19 8240 0014 7163 0010 8772 0011 
18 8226 0012 7153 0056 8761 0014 
17 8214 0015 7097 0014 8747 0011 
16 8199 0014 7083 0072 8736 0010 
15 8185 0016 7011 0050 8726 0012 
14 8169 0022 6961 0017 8714 0018 
13 8147 0040 6944 0042 8696 0015 
12 8107 0028 6902 0021 8681 0022 
H 8079 0023 6881 0025 8659 0018 
10 8056 0021 6856 0036 8641 0024 
9 8035 0029 6820 0056 8617 0021 
8 8006 0044 6764 0073 8596 0036 
y. 7962 0062 6691 0168 8560 0030 
6 7900 0062 6523 0072 8530 0043 
5 7838 0066 6451 0092 8487 0077 
4 7772 0068 6359 0098 8410 0082 
3 7704 0147 6261 0087 8328 0096 
2 7557 0291 6174 0228, 8232 0193 
t 7266 5946 8039 


Note.—In this table the figures are given correct to four decimal places and the decimal points 


are omitted. 
a AR? = Ra? — Rej’, where j = i +1. 


594. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


the three conditions as the number of rater-clusters was sequentially 
and systematically reduced from 38 to a single all encompassing 
cluster. Table 7 shows a noticeably higher predictive efficiency 
(within-rater consistency) for the raters under Condition Dg as 
opposed to those of Condition Dz, while the Condition T raters’ 
predictive efficiency falls somewhere between those of the other two 
conditions. Also, note that the between-rater consistency, which is 
measured by the overall drop in R? from the 38-system stage to 
the single-system stage or the sum of the AR values, varies be- 
tween conditions, where it is lowest (.084) for Condition Dg, high- 
est (.165) for Condition Dr and between the previous conditions 
(.119) for Condition T. 


Analysis of Variance on Policy Similarity 


The data used to make the policy comparisons were obtained 
from the two-group clustering stage (system 2), that is, the raters’ 
regression equations were clustered into two distinct groups within 
each condition. These clusters maximized the between-group differ- 
ence and the within-group similarity of the regression equations. 
Table 8 summarized the results of an unweighted means analysis 


TABLE 8 
Unweighted Means Analysis of Variance Source Table 
Source df MS F P 
Between Subjects 113 
C (Conditions) 2 12.3270 1090.88 <.001 
G (Groups) 1 0323 2.86 <.10 
c XG 2 .0015 13 
$'s/G 108 0113 
Within Subjects 570 
(Traits) 5 1.9500 90.70 .001 
z xC 10 .4037 18.78 -001 
XG 5 2075 9.65 <.001 
TXCxG 10 0273 1.27 <.25 
T X S’s/G 540 0215 k : 
Total 683 


Note.—For an unweighted-means solution, s 
the AN i S8ro401 7 SSyare +SS, -In 
i nies model Conditions, Groups, and Traits ‘were treated rs Sized ‘variables, Bubjecta ‘no 8 


of Variance for unequal group size performed upon the validity co- 
efficients of the predictor traits for each rater in order to examine 
the results for possible between-cluster differences, between-condition 
differences, and for cluster-by-condition differences, 


— 
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Between Profile Comparisons. The analysis revealed that the 
main effect of Conditions was significant at p < .001, but no sig- 
nificant Groups effect was found. Also, no significant (C X G) in- 
teraction was found. 

The significant Conditions effect indicated that the policy levels 
for the three conditions were significantly different. The pattern of 
the vectors of average validities for the raters under each con- 
dition are shown in Figure 2, which indeed depicts the raters for 
the three conditions as holding somewhat similar policies but at 
substantially different average levels. 

The nonsignificant Group effect (F = 2.86, df = 1,108) indicates 
that the policy levels for the two JAN subpolicy groups were not 
significantly different when averaged across all three conditions. 
In addition, since no significant interaction (F = .13, df = 2,108) 
was found between Conditions and Groups it would appear that 
similar group subpolicy levels emerged from each condition. A more 
comprehensive clarification of these results can be obtained by exam- 
ining Figures 3, 4 and 5. From these figures it can be seen that the 
two subpolicies (profiles) obtained by JAN under each condition 
do not appear to have different average heights—thus, the non- 
significant Groups effect. Also, since the difference in profile heights 
or levels of the two groups does not appear to change over the three 
conditions the nonsignificant C X G was not an unexpected finding. 

Within Policy Comparisons. The main effect of Traits and the 
two interactions, Traits by Conditions and Traits by Groups, were 
all significant, but no significant triple interaction (Traits by Con- 
ditions by Groups) was found. The significant Traits effect (p < 
.001) implies that the raters placed differential emphasis upon the 
six job traits as measured by the differences in the average validity 
coefficients. It would further imply that the raters were able to 
discriminate, and indeed did discriminate, between the job traits. 

The significant Trait by Condition interaction (p < .001) is 
probably the most critical index, since it means that dissimilar 
policies emerged from the three conditions, that is, the job traits 
received differential relative emphasis within the three conditions. 
In other words, it demonstrates the differential significant influ- 
ence which the traits, under the different conditions, had upon the 
raters as reflected in their policies based upon validity coefficients. 
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aN is shown most dramatically in Figure 2 where one can see that 
ahi patterns for the three different conditions differ greatly 
RO exaggeration even though the basic patterns are themselves 

milar. Thus the raters in Condition Dz, the orthogonal stimulus 
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Average Validity Coefficient (7) 


l 2 3 4 5 6 
Job Traits 


Figure 3. Average validity coefficients of the six predictor traits for Condition 
T raters (two subpolicy clustering stage). 


dimension situation, exhibited much larger differences between the 
job traits than did the raters within Condition Dg, the highly cor- 
related stimulus dimension situation, while it would appear that 
the Condition T raters fell somewhere between those of the previ- 
ous two conditions. 
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Figure 4. Average validity coefficients of the six predictor traits for Condition 
Dz raters (two subpolicy clustering stage). 


The significant Trait 
that the traits were di 
JAN subpolicy groups, 


by Group interaction (p < .001) indicates 
fferentially conceptualized within the two 
but not also across the conditions as the 
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Average Validity Coefficient (F) 


Job Traits 


Figure 5. Average validity coefficients of the six predictor traits for Condition 
Ds raters (two subpolicy clustering stage). 


case would have been had the triple (T X C X G) interaction been 
significant. From Figures 3-5 it can be seen that although disparate 
rater subpolicies exist for the two groups (significant T X G inter- 
action), a similar pair of JAN subpolicy groups emerged in each of 
the three conditions; thus, the nonsignificant T X C X G in- 
teraction. 


PROF Analysis 


Finally, the PROF procedure (Wherry and Naylor, 1966) was 
employed on all profiles. This involved a principal axis factor anal- 
ysis followed by a varimax rotation using the R matrix obtained 
by intercorrelating the validity vector for each rater with the 
validity vectors of all other raters across the combined three con- 
ditions. 

There were five factors which emerged from the factor analysis. 
Examination of these factors indicated that the first four factors 
could probably best be interpreted as “condition” factors. This is 
clearly seen by looking at Table 9, which was developed by rank 
ordering the twenty highest factor loadings for each factor regard- 
less of the condition. The results show that factors 14 are essen- 
tially “condition” factors in that they were defined almost ex- 
clusively by raters from conditions Dz, Dp, T, and again Da 
respectively. Factor E was a hybrid. The logical implication being 
that the strategies obtained under the different conditions are not 
similar—rather they tend to represent independent policies. 
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Discussion and Conclusions 


The results generally support the contention that variations in, 
or deviations from, a TRUE cue R matrix can and do significantly 
affect a rater’s performance. This conclusion has important impli- 
.cations, particularly for those situations where artificial, rather 
than “real” stimulus objects are employed, or indeed, must be 
employed. 

The most impressive evidence which supports the conclusion that 
deviations from a TRUE cue R matrix do significantly affect a 
rater’s judgment policy evolved from the analysis of variance, in 
that the Trait by Condition interaction was found to be significant. 
This significant T  C interaction implies that the rater policies, 
measured by validity coefficients, are different in different ways 
within the three conditions, that is, that the policy profiles across 
the six traits for each condition departed significantly from parallel- 
ism. This conclusion is further supported by the interpretations at- 
tached to the significant main effects of Conditions, which implies 
that the policy levels for the three conditions, measured by the mag- 
nitudes of the average validities for the traits, are different. Thus 
both profile pattern and profile level seem to be affected by R 
matrix structure. 

Further evidence of the effect of Conditions upon judges’ per- 
formance can be seen from the behavior of other indices, such as 
R? and Rê. For instance, it is evident that the stimuli used in all 
conditions resulted in highly consistent intrajudge performance as 
depicted by the large R? values for all raters, which in turn, denote 
a high degree of success in capturing rater’s policies. However, de- 
spite the fact that most R? values were large, there exist systematic 
differences between these values across the three conditions. These 
differences may have resulted from the fact that while raters within 
the orthogonal condition (Dz) were able to discriminate clearly and 
meaningfully between the predictor variables (job traits), at the 
same time they found the decision making task difficult since each 
trait must be attended to separately. This would cause the rater 
R? values to drop, as would be expected. On the other hand, the 
raters within the highly correlated condition (Dg) were less able to 
discriminate, but found the decision making task easier, resulting 
in high rater R? values. 
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Also, the composite R? values (R2) obtained from the sequential 
stages of JAN were found to be relatively large indicating that even 
when all the raters within a condition were clustered into a single 
composite, a high level of predictability remained. In addition, it 
implies the individual raters’ policies within a condition were rela- 
tively homogeneous, a fact borne out by the factor analysis. How- 
ever, the systematic differences between R,? values across conditions 
simply reflect the difference previously explained pertaining to the 
individual raters’ R? values. 

The PROF factor analysis also yielded rather conclusive data 
supporting the effect of R matrix distortion. The factor analysis as 
performed had the chief advantage that the similarities or dis- 
similarities of factor-strategies across conditions could emerge, 
since all raters were included in a single analysis, This allowed for 
the likelihood of obtaining “condition” factors, as was borne out in 
the results. Indeed, the first four factors were clearly defined by 
raters from specific conditions. 

The implications of these data seem rather straightforward. If 
one is interested in obtaining information about the judgmental 
policies of individuals toward a certain class of stimulus objects he 
needs to be certain that the underlying cue R matrix for his experi- 
mental stimuli is representative of the R matrix describing the 
population of stimuli of which his sample is assumed to be a sub- 
set. Otherwise, his ability to generalize is obviously going to be 
limited. 
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THE TEST-RETEST RELIABILITY OF CHILDREN’S 
RATINGS ON THE SEMANTIC DIFFERENTIAL! 


FRANCIS J. DI VESTA 
AND 
WALTER DICK? 
The Pennsylvania State University 


Tue present study of the reliability of the semantic differential 
technique was made to supplement the data from recent investiga- 
tions (Di Vesta, 1964) providing positive evidence regarding its use 
with children. Several kinds of reliability might be adequately 
studied. Among these are item, factor-score, and concept-meaning | 
stability. Osgood and his associates (1957) report highly accept- 
able levels of these various forms of reliability when the technique 
is used with adults. Jenkins, Russell, and Suci (1958) also report 
coefficients of stability of .90 or higher for mean scale values and 
mean profiles of concepts based on ratings made by 20 or more col- 
lege students. However, comparable studies of semantic differential 
reliability, when used with children, are not known to the authors. 
Data regarding the reliability at several grade levels should pro- 
vide useful guides to other investigators in determining the ade- 
quacy or limitations of the semantic differential for applications 
with individuals or groups of children in the elementary grades. 

Since the semantic differential yields a profile of ratings, a num- 
ber of different scores may be used in determining the reliability of 


1 The study reported here was supported by Grant Number MH2900 from 
the National Institute of Mental Health and Grant Number HD-00872 from 
the National Institute of Child Health and Human Development, United 
States Public Health Service. The data were collected at Syracuse University. 
All computations were made by the Pennsylvania State University Comput- 
ing Center. 

2 Now at Florida State University. 
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this instrument. Its stability can be determined by the test-retest reli- 
ability coefficients calculated for scale items, factor scores, dis- 
tances (D.) between concepts, and polarity, that is, distances 
(D,) of ratings from the origin of the semantic space. These, in 
turn, may be based on individual or mean ratings. Since the reli- ~ 
ability of distances between concepts is dependent on the reliability 
of the factor scores on which they are made, the decision was made 
to limit the present analysis to test-retest reliability of scales, to the 
deviation of factor scores from pretest to posttest, and, where appro- 
priate, to test-retest reliability of factor and polarity scores, sepa- 
rately based on individuals and means. The reliability of the semantic 
differential was studied separately for delayed and immediate retests. 


Method 


The data for the present study were gathered as part of three 
factor analytic studies of the development of children’s meaning. 
Retest data were gathered in two of these three independent studies. 
Since the subjects, selection of scales, selection of concepts, and 
method have been described in detail in an earlier publication (Di 
Vesta, 1964) only the procedure related to the reliability study 
will be presented here, 

In both studies, the reliability of individual scales and of factor 
Scores are presented. Although a number of scales were aligned on 
each factor, only those two scales with the highest loadings were 
averaged to obtain each factor score. The factors, and the scales by 
which they were defined, are as follows: Evaluation . . . good-bad 
and friendly-unfriendly ; Potency . . . strong-weak and brave-not 
brave; Activity . . . fast-slow and moving-still; Size .. . big-small 
and long-short; and Warmth . . . hot-cold and wet-dry. These fac- 
tors were identified through rotation of the principal components 
factors by the Equamax routine to achieve simple structure. 


Delayed Retest 


In the main study (identified as Study II, in the earlier report, 
of which the delayed retest study was a part), 100 Ss in each of 
the grades two through seven rated 100 concepts on 27 scales. Each 
S rated 20 concepts. Except for tatings made by Ss in the seventh 


DI VESTA AND DICK 607 


grade, all ratings were over a three- or four-week period in order to 
accommodate the convenience of the school personnel. The ratings 
by the seventh grade Ss were made during a three-day period. After 
a one-month interval, a graduate assistant asked each S in each class 
to rate again one of the concepts (selected at random) previously 
rated. Each S rated a concept different from that of the other Ss 
within his group. 

A total of 522 Ss are represented in this study: in grade two, 
N = 86; in grade three, N = 87; in grade four, N = 91; in grade 
five, N = 73; in grade six, N = 89; and in grade seven, N = 96. 
The number of concepts rated corresponds to the number of Ss in 
each grade. Although more than 100 Ss were originally tested, 
something less than this number was finally obtained because of 
absences during either the test or retest period. In addition, a few 
papers were rejected in each class because of incomplete responses 
or because the papers were obviously pattern-marked. 


Immediate Retest 


In the main study in which the immediate retest was adminis- 
tered (identified as Study III in the earlier report) 100 additional 
concepts in sets of 20 concepts were rated by Ss in grade three 
(N = 194); grade five (N = 129); and grade seven (N = 181). 
These Ss were in a different school system than those used for the 
delayed retest. Each S rated 20 concepts, selected at random, on 
21 scales, in two series of ten concepts each, during a period of 
three to five days. The eleventh concept in the second series was a 
repeat of one, selected at random, previously rated in that series. 
The concept in the retest was administered separately from the 
other concepts. Some Ss recognized that the concept had been pre- 
viously rated, but were always instructed that the rating was to be 
made independently of the earlier rating. Each of the 50 concepts 
was rated by three to five Ss. Thus, although the N was small, we 
were able to use mean scores as well as individual scores for the 
analysis, As a result of absences at the time the original test or 
the retest was administered, a total of 488 Ss were actually used in 
the immediate retest study: N = 192 in grade three, N = 123 in 
grade five, and N = 173 in grade seven. Scores were obtained on 
50 concepts but because of absences one concept is not represented 
in each of the grades three and five. 
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Delayed Retest 


The data for this study are based entirely on comparisons of in- 
dividuals. Each individual’s rating is of a concept different from all 
other individuals in his group. The interval of time varied from the 
original (test) to second (retest) rating of the concept. In general 
the period between the two ratings was roughly four weeks. 

The individual test-retest scores on each scale were correlated, 
In Table 1 it may be seen that all of the correlation for scales are 


TABLE 1 
Correlations between Test-Retest Scale Ratings 


Made by Individuals: Delayed Retest 
e eee 


Grades 


i 

EJE 
R 
ab 
Bale 


37 66.59 

Round-Square 31 33 34 34 46 54 
Sweet-Sour 22 Al .52 .57 .77 -71 
Dark-Light Pee E 60. «643 (CSL 
Little-Big 53 .37 38946898 
First-Last 18 22 20 .438 0 87.65 
Wet-Dry .33 aR 71 O 30 45 
Not Brave-Brave .39 at ee ee 
Ugly-Pretty -13 55 470.65 52 58 
Light-Heavy AT 34 8.40 1.389 47  .63 
Make Believe—Real -33 34° 48258 00 
Sad—Funny -19 39° 5465 45 63 
Still-Moving .31 35.57) 668 l 8 
Good-Bad -33 -39 69 85 .68 75 
Quiet-Loud -50 -26 «56 38 .46 68 
Cold-Hot .34 vel S Ce ON ee 
Smooth-Rough +25 29 45 (14452 58 
Friendly-Unfriendly 141 -383 62 69 68 63 
Strong Weak .32 25 54 53 173 49 
Red-Blue -46 130° 121005148 tC AT 
Tight Loose -25 130 142) 6101914 
New-0la AT 28 8.31 «6431s 
Eh i = Don 86 ől 

$ x a e 81 47 
Soft-Hard ii 2 -62 66 45 Pi 

a i ` a 27 o. 
Same-Different -20 09.4881 16846 
low, 


> ranging from an Average correlation, across grades, of .27 to 
- 4. progressive increase in the magnitudes of the coefficients for 
grades averaged over scales is reflected in the range, from an 
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average of .33 in the second grade to an average of .55 for the 
seventh grade. The largest correlation coefficients are to be found 
for the Evaluation scales followed in magnitude by the scales for 
Potency and Activity. The lowest correlations are for those scales 
in the Warmth factor and a composite of other scales including same- 
different, dull-sharp, and tight-loose. 

A second analysis was based on the factor scores obtained by 
simply adding the ratings on two scales associated with each factor, 
Two dominant trends are apparent in these data as presented in 
Table 2. The first is a definite increase in reliability of ratings 


TABLE 2 


Correlations between Test-Retest Factor Scores of Ratings 
Made by Individuals: Delayed Retest 


Grades 
Factors 2 3 4 5 6 7 
Evaluation -39 .49 75 -86 -79 -78 
Potency -55 -49 60 -40 64 55 
Activity 51 85 „54 .64 .64 54 
Size .19 .43 54 54 46 -63 
Warmth Al 26 21 61 55 22 


made on the semantic differential between the third and fourth 
grades. The second trend is that the ratings made of the Evalua- 
tion, Potency and Activity factors are, in general, more reliable 
than any of the remaining factors. 

A measure of meaning related to the factor scores is the intensity 
of meaning. However, rather than employing a single score, the in- 
tensity of meaning (polarity) is determined by using the deviation 
of the profile or factor scores from the origin of semantic space and 
is calculated by the formula Da = \/3; dą? where iis the factor score, 
j is the factor and 1 is, in this case, the constant 4 (the mid-point, 
or origin, of the scale). The scores for each of the five factors were 
used in obtaining the polarity index (D,). The reliabilities of the Dp 
scores are presented in Table 3. Since the polarity measure, as an 
index of meaning, is of theoretical significance the means and stand- 
ard deviations of the Ds are also presented in this table. As in the 
previous analysis the higher reliability coefficients are found in the 
upper three grades. Overall, the stability of the polarity measure is 
somewhat higher than for factor or scale scores, and gradually in- 
creases from the second to the seventh grade. The mean polarity 
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TABLE 3 


Correlations, Means, and Standard Deviations of Test-Retest Polarity 
Scores of Ratings Made by Individuals: Delayed Retest 


Pre-Test Post-Test 

Coefficient of — — 

Grade Reliability Mean SD Mean 8D 
2 -50 3.48 1.18 3.17 1.47 
3 .55 3.11 1.10 3.03 1.10 
4 .45 3.10 1.06 2.71 1.32 
5 -62 3.20 1.23 3.03 1.11 
6 .62 3.29 1.10 3.14 1.16 
7 -73 3.24 1.20 3.01 1.11 


index is lower on the retest at every grade-level than it is on the 
original test. 

Another indication of reliability of interest in the application 
of the semantic differential is the determination of the probability 
of obtaining given absolute deviations of factor scores from test to 
retest. The percentages of responses giving each deviation (per 
cent) and associated probability (P) were computed for five fac- 
tor-score averages by grade level.? Since each score was obtained by 
averaging over two scales, deviations were in terms of half scale 
units. There were few appreciable differences among grades in these 
data. In general, the results correspond to those presented by Os- 
good, Suci and Tannenbaum (1957) who used a larger number of 
scales in obtaining factor scores, 


Immediate Retest 


The relatively low reliability coefficients for the first study may 
be attributed to the interval of four weeks between test and retest 
and to the fact that only individual ratings were used. These con- 
siderations are of importance since children’s concepts are continu- 
ally undergoing change and instruction in school may have an ap- 
preciable influence on the ratings. In addition, data from correlated 
seales of the semantic differential are generally pooled to obtain 
factor scores which Tepresent the judgments of individuals or 
groups. The present analysis was undertaken to determine the ef- 
fects of combining scale scores compared to the effect of using in- 


eee 
2 sae . sae im: 
The deviations and associated probabilities for the delayed retest and im- 


retest studies are presented in Tables A 

: i À and B. They have been 

BOOD, reiting SS er eal eimentation Tastitute, Onder Document No: 
> ‘or 35-mm. microfilm or $1.25 for 6 by 8 in. photocopies. 
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dividual scores as in the previous analysis. This study also attempts 
to examine the stability of the semantic differential over a short 
period of time. 

While our procedures permitted us to obtain concept means based 
on groups of three to five individuals, it still is not the most opti- 
mal procedure. In studies (Jenkins, Russell and Suci, 1958; Miron, 
1961) employing adult Ss, such group means are based on 20 or 
more Ss. Hence, the estimates of reliability presented here must be 
considered as conservative estimats of the stability of children’s 
ratings. The period between the test and retest was one or two days. 
All methods of analyzing data for the first (delayed retest) study 
were followed in the present study. Judgments of concepts were 
made on 21 scales (shown in Table 5) rather than 27 as in the first 
study. The five factors of Evaluation, Potency, Activity, Size, and 
Warmth are defined by the same scales described above and so are 
directly comparable. 

The correlations of test-retest ratings on each scale over all in- 
dividuals and concepts within a grade are presented in Table 4. 


TABLE 4 
Correlations between Test-Retest Scale Ratings 
Made by Individuals: Immediate Retest 


Grades 
Scales 3 5 7 
Heavy-Light 74 -72 .86 
Quiet-Noisy 66 64 .79 
Dry-Wet 42 -60 67 
Hot-Cold 38 .35 63 
Make Believe-Real 52 .65 .52 
Terrible-Wonderful .65 .74 -76 
Smooth-Rough .62 .49 .62 
Powerless-Powerful 55 70 .65 
Usual-Strange 50 51 72 
Unfriendly-Friendly 62 53 .69 
Moving-Still 56 -60 .66 
Strong—-Weak -63 53 .60 
Good-Bad 74 64 81 
Fast-Slow 43 .39 .58 
Old-New 32 -50 -66 
Soft-Hard 62 -66 79 
Little-Big -58 -69 71 
Straight-Curved 70 -58 61 
Dark—Light 41 -35 62 
Long-Short 39 84 53 
Brave-Not Brave 67 -70 61 
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Compared to the delayed retest data, reported above, all coeffi- 
cients are considerably higher in the present study. The coefficients 
for grades, averaged over scales, are .56 for each of the grades three 
and five, and .67 for grade seven; the coefficients for scales (aver- 
aged across grades) range from .42 to .77. Relative to the magni- 
tude of the coefficients for the other factors those for Evaluation, 
Potency, and Activity are found to be the most reliable ratings; 
and judgments made on the hot-cold, old-new, dark-light and long- 
short scales are less reliable. These results are consistent with those 
reported for the delayed retest. 
The factor scores were averaged over ratings, made by groups of 
three to five Ss, of each concept thus yielding factor scores for con- 
cepts. Correlations between concept scores on the test and retest 
administrations were then obtained. The factor scores for each in- 
dividual were also correlated, as in the delayed retest study. These 
data are presented in Table 5. All correlations for concepts are of 


TABLE 5 
Correlations between Test-Retest Factor Scores of Individuals 


and Concepts (Groups): Immediate Retest | 
a a a e o o j| 


Grade 3 Grade 5 Grade 7 | 
tees T 
Factor Concepts Individuals Concepts Individuals Concepts Individuals 
Evaluation -89 17 .87 78 84 
Potency 79 ‘67 79 ‘69 ‘Ba 2 
Activity -80 .62 81 64 73 -69 
Size .82 64 -67 .64 .89 -75 
Warmth -60 .38 -70 .64 .85 -68 
Ric oo ee S S E A 


acceptable magnitude and in no instance was the correlation for 


concepts lower than the respective correlation for individuals. All 


correlations are sufficiently high to indicate the functional utility of 
separate factor scores, based on only two scales, for research pur- 
poses. 
; The reliabilities of polarity scores were also obtained for both 
individuals and concepts. The coefficients of reliability, thus de- 
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TABLE 6 


Correlations, Means, and Standard Deviations of Test-Retest Polarity Scores 
of Ratings Based on Individuals and Concepts (Groups): Immediate Retest 


Pre-Test Post-Test 
Coefficient of — — 
Reliability Mean SD Mean SD 
Concepts 

Grade 3 .82 3.70 .14 3.35 0.9% 
Grade 5 -69 3.91 1.19 3.72 1.21 
Grade 7 .89 3.85 1.12 3.83 1.11 

Individuals 
Grade 3 74 4.65 1.52 4.53 1.58 
Grade 5 -67 4.58 1.21 4.54 1.29 
Grade 7 -79 4.53 1.22 4.54 1.27 


The tendency, noted earlier, for the polarity of judgments to de- 
crease on the second testing prevails. However, a small general in- 
crease in polarity (meaning) with age in the concept means is also 
to be noted. 

Absolute deviations from the test and retest administrations were 
also computed. The percentages of individuals whose ratings did not 
change from test to retest were substantially higher in the “no devia- 
tion” range than in the delayed retest study. Otherwise, there were 
no appreciable differences among grades. However, even with these 
data we can expect about 10 to 15 per cent of the ratings to deviate 
by more than two seale units. i 

Discussion 

The findings in the present study clearly suggest that over brief 
periods, the semantic differential is a reliable instrument when used 
with children. The coefficients of stability are comparable to those 
ordinarily obtained by other techniques requiring judgments by in- 
dividuals. In addition, it can be used with children as young as 
those in the third grade, and perhaps those in the second grade with 
little change in the procedures used in administering the scales to 
adults. However, with children in the lower grades it is advisable 
to use larger numbers of Ss and, perhaps, three or more scales in 
arriving at factor scores. The reliability of the data generated by 
the semantic differential is higher when ratings from two scales are 
combined to obtain factor scores than when only single scales are 
used. There is considerable variability in the reliabilities of individ- 
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ual scales and in many cases they are unsatisfactory. The extent to 
which such low coefficients reflect instability of the measure or of 
the concepts being judged can be determined only by further in- 
vestigations. Warr and Knapper (1965), for example, found that 
test-retest reliability depends on the concept being judged, especially 
where metaphorical extensions are involved. For usual research 
practice, where the concern is with only one to three dimensions, it 
seems likely that the investigator will choose to combine the scores 
from three to five scales in order to realize higher stability. 

As with even the most standardized psychometric instruments, 
the stability of the ratings on the semantic differential is greatly re- 
duced when the interval between test and retest is increased. This 
seems especially so for children in the second and third grade. 
Whether instability in the lower grades, over a one-month interval, 
is due to real change in concept meaning or merely reflects error 
variance must await further investigation, the procedures for which 
are suggested by Osgood et al. (1957). However, the results of our 
second study suggest that the instability of ratings by children in 
the lower grades may be the result of actual change in concept 
meaning, that is, the ratings by third grade children are very stable 
when test and retest administrations are spaced one to two days 
apart. 

The reliability of the ratings of individual scales, and of the 
factor scores, also indicates the saliency of the familiar Evalua- 
tion, Potency and Activity dimensions. These appear to be general 
and pervasive factors. It seems likely that they are experienced 
early in the development of the child. That other, somewhat less 
salient and less pervasive, factors exist in the child’s language has 
been demonstrated in our earlier studies. That they can be meas- 
ured reliably is demonstrated in the present study. 

The absolute deviations of the factor scores from test to retest 
are approximately the same as those found with adults by Osgood, 
et al. (1957) with little appreciable difference among grades. How- 
ever, the probability of a change in factor scores of more than 2.00 
scale units would occur 10 to 15 per cent of the time with the indi- 
vidual ratings of children compared to a probability of .05 for a 
change in factor scores of more than 1.00 to 1.50 for data based on 


individual adult ratings where the test-retest interval was thirty 
minutes. 


| 
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Summary and Conclusions 


The stability of the semantic differential technique was examined 
under delayed and immediate test-retest conditions. The study was 
conducted with children in grades two through seven in the former 
and in grades three, five, and seven in the latter condition. There 
were 522 Ss in the delayed retest study and 488 Ss in the immediate 
retest study. 

The coefficients of stability were based upon the correlation of 
factor scores derived by simply averaging the ratings of two scales 
with high loadings on each of five factors in the delayed retest study 
and seven factors in the immediate retest study. In general the se- 
mantic differential technique was shown to be an acceptably stable 
instrument when used with children as young as the third grade un- 
der immediate retest conditions. Reliability for concepts based on 
group means is higher than that for individuals even when groups 
are composed of only three to five subjects. An extrapolation of 
these results suggests the possibility that levels of reliability of 
concept scores of children will approach those obtained for adults 
if 15 to 20 Ss are used in the rating procedures. The lower reliability 
coefficients obtained for the delayed retest condition were attrib- 
uted in part to changes in meaning of concepts although the ex- 
istence of larger measurement errors was also considered as a pos- 
sible contributing factor to the lower reliability of ratings made 
by second and third grade children. 

The individual scales and the factor scores relating to the salient 
dimensions of Evaluation, Potency, and Activity were found to be 
more reliable than the remaining scales under both delayed and 
immediate retest conditions. Used with caution, the factor scores 
of Size, and Warmth were sufficiently reliable for research purposes 
and suggest other dimensions for further investigations in which 
the semantic differential technique is employed. 


REFERENCES 
Di Vesta, F. J. A Developmental Study of the Semantic Structures 
of Children. Technical Report No. 8. Research Grant No. HD- 
00872, National Institute of Child Health and Development, 


1964. 
Jenkins, J. J., Russell, W. A., and Suci, G. J. An Atlas of Semantic 


616 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Profiles for 360 Words. American Journal of Psychology, 1958, 
71, 688-699. 

Miron, M. S. The Influence of Instructional Modification upon 
Test-retest Reliabilities of the Semantic Differential. EDUCA- 
TIONAL AND PSYCHOLOGICAL MEASUREMENT, 1961, 21, 883-893. 

Osgood, C. E., Suci, G. J., and Tannenbaum, P. The Measurement 
of Meaning. Urbana, Illinois: University of Illinois Press, 1957. 

Warr, P. B. and Knapper, C. Some Factors Affecting the Reliabil- ` 
ity and Validity of Semantic Differential Scales. Aberdeen, 
England: Paper read to the British Psychological Society, 
April, 1965. 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
1966, 26, 617-622, 


EVIDENCE ON PROBLEMS IN ESTIMATING 
COMMON FACTOR SCORES 


JOHN L. HORN ax» WILBUR C. MILLER 
University of Denver 


Wirr the advent of the computer in psychological research, 
problems in estimating factor scores have come increasingly to the 
fore. In a recent article Horn (1965) compared various such esti- 
mation methods in terms of their theoretical-mathematical prop- 
erties and by intercorrelating the scores estimated by the different 
procedures. His findings showed that the scores estimated by three 
complete methods were highly similar, as were those estimated by 
three so-called incomplete methods, but that these two broad 
classes of techniques yielded scores which correlated only .80 on 
the average. He found, also, that the intercorrelations among dif- 
ferent orthogonal factors estimated by each of the three complete 
methods averaged .08 (zero being the desired value), whereas these 
intercorrelations for the incomplete methods averaged .14. 

This kind of information no doubt has relevance for questions con- 
cerning the usefulness of these various techniques in research and ap- 
plied work. But an essential item of information is lacking. It is 
important to know not only how the factors estimated by the dif- 
ferent procedures are interrelated among themselves, but also how 
each relates to the variables upon which a factor analysis is based. 
One purpose of the present investigation is to supply this kind of 
information. 

The data upon which Horn’s (1965) analyses were based were 
somewhat atypical. At least they were not representative of those 
found in studies where a strong positive manifold prevails among 
the correlations for variables. On first consideration this might 
seem to be a rather minor matter. But when one stops to take ac- 
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count of the fact that (at least with some methods of estimation) 
this can imply that all variables will be given positive weights in 
the equations for estimating two supposedly independent factors, 
then it becomes apparent that with certain kinds of data these 
methods could yield factor estimates which were much more highly 
intercorrelated than would be implied by the factor solution. For 
example, if factor scores are estimated by adding standard scores 
(for all variables) weighted by the factor loadings of the variables 
(Horn’s method 4) and the factor loadings are all positive for two 
factors (these being cooperative: cf. Cattell (1957) ), then all that 
distinguishes the two sets of factor scores is the pattern of positive 
differential weights. But if there are many variables, then, as 
Richardson (1941) showed analytically and as Guilford, Lovell 
and William (1942) found empirically, these two linear com- 
posites are apt to correlate quite highly (i.e., unless the patterns of 
, weights are very different indeed). Cooperative factors are not 
likely to be defined by widely different coefficients. Hence under 
these conditions factor score estimates are not likely to be as in- 
dependent as the factor solution implies, 

Independence would tend to be maintained with the complete 
methods, on the other hand, since in these cases, as in the usual 
multiple regression equation, if one variable of a highly correlated 
pair 1s given a positive weight, the other is likely to be given a 
widely different negative weight. 

Horn’s (1965) data did not show the high degree of positive 
manifold here stipulated. It is not surprising, therefore, that he did 
not find this effect. The intercorrelations among different factors 
were low for all of the factor estimation methods he studied. But 
would these correlations be low in all cases if a high degree of posi- 
tive manifold obtained among the intercorrelations for variables? 
A second purpose of the present investigation is to examine this 


question by analyses with data having the characteristics here 
specified. 


Procedures and Results 


A series of 20 teacher ratings was 
children. Correlations between these 
subsample of 167 children for which 
These scales, 


obtained on a sample of 195 
scales were determined on a 
complete data were available. 
although measuring what were supposed to be inde- 
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pendent attributes—such as physical skill, creativity, leadership 
ability, sense of responsibility, ete—nevertheless yielded correla- 
tions showing a very substantial positive manifold (this, of course, 
reflecting in part the halo phenomenon so often observed in be- 
havior ratings). Five principal axes factors were found to ac- 
count for the major portion of the reliable, common variance 
among these correlations. These factors were rotated to a position 
satisfying Kaiser’s (1958) Varimax criteria, Then, using the total 
sample of 195 children, the following procedures for estimating 
factor scores were employed: 


W F, = ZRA 

(2) F, = ZR"B 

(8) Fe = ZA 

(4) F. = ZB 
where the F; (subscripts chosen to be consistent with Horn, 1965) 
are N by m matrices of factor scores, Z is the N by n matrix of 
original standard scores, R- is the n by n inverse of the matrix of 
intercorrelations among variables, A is an n by m matrix of factor 
coefficients (i.e., a structure) and B is an n by m matrix in which 
unity is substituted for each “salient” coefficient in A and zero is 
substituted for every other loading. (A variable was defined as 
salient for a factor if it had its highest loading on that factor. In 
all cases this meant that the salient had a loading of at least 45 
on the factor in question and frequently the loadings were higher 
than this. A variable was defined as salient [in this sense] for one 
and only one factor.) The factor scores estimated in these ways 
were intercorrelated, whence the results summarized in Table 1 
were abstracted. 

Table 1(a) shows results comparable to those obtained by Horn 
(1965). That is, Horn had found—with his larger samples of vari- 
ables—that the average correlation for the same factor estimated 
by procedures F, and Fe was .79; for procedures F, and F; this was 
.82 and for procedures F4 and Fe it was .90 (procedure F; was not 


1 These intercorrelations, the factor structure (pattern) matrix and sets 
of intercorrelations to be mentioned in later sections of this paper, have been 
deposited with the American Documentation Institute. They can be obtained 
by remitting $1.25 for microfilm or $1.25 for photoprints (check payable to 
Chief, Photoduplication Service, Library of Congress) for Document No. 8909, 
ADI Auxiliary Publications Project, Photoduplication Service, Library of 
Congress, Washington, D. C. 20540. 


620 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


TABLE 1 


Average Correlations between Factor Scores Estimated 
by Various Procedures 


(a) 
For the Same Factors Estimated by Different Procedures 


Procedure 
F, Fy Fs 
Fı 656 -651 -818 
Fy +259 572 
Fi .918 


(b) 
For Different Factors Estimated by the Same Procedure 


Procedure 
Fı Fy Fs 
—.217 -902 642 


Fı 
.127 


used); in the present study these values are .82, .65 and .92 re- 
spectively. But in Table 1(b) results quite in contrast to the earlier 
findings are shown. The correlation between different, supposedly 
independent, factors estimated by the F, and Fe procedures are 
much higher than would be desirable in most studies. This out- 
come results because all the variables are positively correlated and 
the weights which are assigned in the estimation equations are not 
widely different. As noted before, this outcome is predicted by 
Richardson’s (1941) early theoretical analysis. It agrees with the 
results obtained by Guilford, et al. (1942). 


TABLE 2 
Average Discrepancies between Correlations (For Variables and Factor Score 
Estimates) and Factor Structure Coefficients (Loadings) 
eee ee 


Factor Score Estimation Procedure 
ERE SECA EVEN AE Se 


Fı F, Fy Fe 
hase: SL e 
verage for: 
1. Salient Variables 104 498 147 204 
2. Non-salient Variables -089  —.218 |434 — ‘297 


Table 2 gives a summary of results which permit of some com- 


parisons between estimation procedures in terms of the accuracy 
with which the method gives scores that correlate with variables 
in the way suggested by a factorial solution. A discrepancy larger 
than zero indicates a failure of the estimation procedure. Table 2 
shows that Fı, F4 and Fe give factor scores that are somewhat 
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biased in the direction of having a higher correlation with vari- 
ables than would be suggested by the factor structure, whereas the 
scores obtained by method Fz correlate consistently lower than 
would be suggested. The results follow the same pattern for both 
salient and non-salient variables. Correlations which are larger 
than expected are no doubt so in part because of autocorrelation, 
since the variance of the variable is included in the factor score 
with which it is correlated. The smaller correlations resulting from 
use of method F; reflect the fact that in making the least squares 
estimation, suppression weights are built in (by taking the inverse 
of R). When these are used with all variables (and all factor struc- 
ture coefficients), the resulting factors correlate as required with 
variables (if corrections for autocorrelation are made, the discrep- 
ancies for method F, reduce practically to zero). But when used 
with only a few of the variables in method Fz, over-suppression re- 
sults and the factor score correlations with variables are thus re- 
duced. 

The general findings from this investigation thus argue for a 
position in opposition to Horn’s (1965) implicit contention that 
there is relatively little to choose between the different procedures. 
If there is a rather solid positive manifold among the intercorrela- 
tions for a set of variables or if, under any circumstances, two 
factors are highly cooperative (even though independent), then use 
of method F is likely to produce factor scores which are too highly 
intercorrelated. In these circumstances one of the complete pro- 
cedures, as here represented by method F, should be used. 


Summary 


Four methods of estimating common-factor scores were com- 
pared for use in situations where consistent positive correlation 
obtains between variables. Analytic considerations suggested that 
under these conditions scores estimated by incomplete methods 
might show higher intercorrelations than would be desirable. This 

' prediction was borne out by empirical analyses. In addition data 
_ were presented which allowed for comparison between factor esti- 
mation methods in terms of the correlations which the resulting 
factor scores had with variables. It was found that when the most 
common least-squares (“complete”) procedure for estimating factor 
scores was used, the correlations between factor scores and variables 
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were very close to those shown in the factor structure matrix, but 
that when other (“incomplete”) procedures were employed, the 
differences between the estimate-variable correlations and the factor 
structure coefficients were larger. These results thus argue for use 
of one of the complete procedures when this is possible. 
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NOTE ON RANK BISERIAL CORRELATION: 


GENE V GLASS 
University of Illinois 


Onm may obtain data to be correlated in many different forms. 
They may be in the form of observations on normally distributed 
variables, n consecutive ranks, a dichotomy, a trichotomy, eto. In 
this paper we shall be concerned with developing a measure of the 
correlation between one variable comprising n consecutive untied 
ranks and a second variable comprising a dichotomy. We shall re- 
gard the dichotomy (scored 0, 1) as having a ranking variable un- 
derlying it, In this regard it is analogous to the assumption made in 
biserial correlation that a normally distributed variable has been 
forced into a dichotomy. Of major interest is a surprising equiv- 
alence which results between the coefficient found in this paper and 
a coefficient due to Cureton (1956). It is to present and prove the 
equivalence of these two coefficients, which though quite different in 
origin and derivation are algebraically equivalent, that this paper 
is being written. 

Derivation of the Rank-Biserial Correlation Coefficient, rb 


We shall assume that Y is a variable composed of the n consecu- 
tive untied ranks, 1, 2, . . „n. Let X’ be a variable of the same type. 
A measure descriptive of the correlation between Y and X’ is Spear- 
man’s rho, the product-moment correlation coefficient between the 
two variables. 

Assume that we do not know X’ but instead we have a (0, 1) 
dichotomous variable which has a ranking variable, X’, underlying 
it. The no persons at 0 on X are assumed to have the ranks 1 through 


1I wish to thank Dr. David E. Wiley for his invaluable assistance at the 
early stages of the work reported in this paper. 
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M on X’; the m persons at 1 on X are assumed to have the ranks 
No + 1 through n on X’. 

One might wish to gain information about the population Spear- 
man’s rho between Y and X’ from the scores on Y and X. There is 
an analogy between this situation and that in which the conven- 
tional biserial correlation coefficient arises, In biserial correlation, - 
Y and X’ are normally distributed random variables and their bi- 
variate distribution is such that Y has a linear regression on X’. 
Suppose in the bivariate population X’ is dichotomized. A random 
sample is drawn in which observations are made on Y and on the 
dichotomization of X’. It is desired to estimate po'y, the population 
Pearson product-moment correlation coefficient, fromthe sample. 
Karl Pearson (1909) proposed a sample estimator of psy ‘which has 
been adopted almost exclusively. It is well known that Pa'y = Bary 
oa'/oy, where Bey is the slope in the population of the least-squares 
regression line of Y on X’ and o denotes the standard deviation of 
a variable. The»standard deviation of Y can easily be estimated 
from the sample, and Pearson showed how ow, could be estimated 
from the dichotomously scored sample. The slope of the regression 
line, Boy, was estimated by the slope of the line passing through 
the two points (0, Yo) and (1, Yı), where Yo and Y; are the means 
on Y of those scoring 0 and 1 on the dichotomy, respectively. This 
line is such that the sum of the squared distances along the Y-axis 
of the sample points from the line is minimal (see Kelley, 1947, 
Pp. 373-4). Substituting the slope of this sample regression line and 
estimates of o and oy for the corresponding parameters in Bory 
¢a'/y yields the conventional biserial correlation coefficient: 

5 < 


w 4 
7 Tis = a= ty , 
‘ 4 é * 
where oe % 4 


8," is the unbiased estimator of o,?, 

p = 1 — @ is the proportion of 1’s on the dichotomy, 

z is the ordinate on the unit normal curve above the (100p)th 
percentile of the distribution. 


The coefficient rj, is an estimator of Pe'y, the population product 
moment correlation coefficient between the two undichotomized var- 
jables. Tate (1955) showed that Tsis is a consistent estimater of poy} 
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thus in the bivariate normal population the product-moment and 
the biserial correlation coefficients are equal. 

Suppose now that Y and X’ are ranking variables, and that X is 
a dichotomous variable derived from X’ in the manner indicated 
in the second paragraph in this section. In the population of indi- 
viduals, the product-moment correlation of the ranking variables 
Y and X’ is the population Spearman rank-order Correlation coeffi- 
cient. From a sample of n persons for which observations on Y and 
the dichotomy X are available, we seek an estimate of this popula- 
tion coefficient. An estimate can be derived in the same manner that 
Tvis is derived as an estimate of p. The desired estimator of the 
population Spearman’s tho from a set of n ranks (observations on 
Y) and n dichotomous observations, X, will be denoted rb. The 
sample value of rb will have the form: 


rb = byes =, 
Yy 


~ 


where 


bs» is an estimate of the least-squares regression coefficient 
of Y on X’, 
Sas is the standard deviation of X’, and 


s, is the standard deviation of iG 
+ s 


It is well known that s»? = s,? = (n? — 1)/12 (see Siegel, 1956, 
p. 203). Hence, rb = bys. Following the estimation procedure used 
in deriving the classical biserial coefficient, we let by» equal the 
slope of the line which passes through the two points (Xo’, Yo) and 
(Xx, Yı), where the bar above the variable denotes an average and 
the subscript to the variable denotes one group of the dichotomy. 
For example, + 


+ 5 
Y= Se Yio/mo, where the subscript 7 ranges over persons. 


The slope of the chosen line is the coefficient rb and is given by 


the equation ca 
Y=, 
rb = a. 
17” 


Yı and Yo are easily calculable from the observed ranks, Y. Xj’ 
and Xo’ are calculated as follows: 
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X! eri ¥ Xu'/n. 
E Xw = 1+2- + my = nol + 1/2. 
Therefore, Xo’ = (n, + 1)/2. 
DY Xa’ = Go +) +o +2) +- 
: + (n +m) = nino + n + 1)/2. 


Therefore, X,’ = m + (m + 1)/2. 
Thus, X,’ — X,' = n/2. 
It follows, then, that rb has the form 


rb= we =2(%,— Py). a) 


The coefficient rb has an attractively simple definitional form. 
Computational formulas for rb can be found which are even simpler. 
These computational formulas follow: 


n- 2[r -24], B 


Formulas (2) and (3) are equivalent. One of the two will probably 
be very much simpler, depending on the sizes of n and no and the 
size of the ranks in one group of the dichotomy. (Note that rb is not 


defined when either nı or ne is zero.) The computation of rb will be 
illustrated on the data below: 


The ranks of those scoring 7 on the dichotomy appear under 
column 1. 


Scores on Y for Calculations 
La X=0 m=4,%m =6 

10 8 Y, = 30/4 

z 6 rb = (2/6)(80/4 — 11/2) = 2/3. 
5 

4 3 
2 
1 


The significance of the difference of rb from zero can be tested by 
an independent ranks test such as the Mann-Whitney U-test. One 
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simply runs the ranks test on the ranks of those scoring 1 on the 
dichotomy against those scoring 0 on the dichotomy. (See Siegel, 
1956.) 


Cureton’s Rank-Biserial Coefficient 


Cureton (1956) developed a coefficient descriptive of the correla- 
tion between a ranking variable and dichotomous variable. We 
shall denote his coefficient by re for the time being. As compared 
with the coefficient developed in the previous section, re makes no 
assumption of a ranking variable underlying the dichotomy, and it 
is calculated by counting inversions and agreements; hence it is 
not a product-moment coefficient. The methods of computations for 
rb and r are quite dissimilar. Oddly enough for untied ranks 
rb and r, always give the same value for a set of scores, This will 
be proved in the following section. In this section the derivation of 
Ta Will be outlined. 

Let X be a dichotomous variable and Y a variable comprising 
the n untied ranks 1, 2,..., n. Cureton sought a coefficient de- 
scriptive of the relationship between X and Y such that (a) it would 
have attainable limits +1 under all circumstances, (b) it would be 
+1 when the n, highest ranks are all 1 on the dichotomy, (c) and it 
would be strictly nonparametric, i.e., defined wholly in terms of in- 
versions and agreements without such concepts as mean, variance, 
regression, etc. 


To compute r, the data are arranged in the following manner: 


Scores on Y for 


x=1 X=0 Agreements Inversions 
10 6 
: ` 
8 2 
7 pea 5 i] 
6 1 
5 1 
4 3 
3 
2 
1 Pea 
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There is an agreement at any given rank under column / for every 
smaller rank under column 0. There is an inversion at any given 
rank under column 0 for every smaller rank under column 1. Thus 
there are three agreements corresponding to the rank “4” under 
column 1 since there are three smaller ranks, 3, 2, and 1, under 
column 0. P is the sum of all agreements in the data, and Q is the 
sum of all inversions. 
Cureton defined r, as follows: 


T, = (P — Q)/Pius: (4) 

When no ties exist (the only case we have considered throughout 

this paper) Pmax = non; again, no is the number of persons at 0 on 
the dichotomy and n, is the number of persons at 1. Hence 


Te = (P — Q)/non. (5) 
For the above data, 


T. = (20 — 4)/4-6 = 16/24 = 2/3. 

Notice that rb and r, have the same value, 2/3, for the above 
data. In the next section it will be shown that rb and r, are alge- 
braically equivalent in the no ties case, even though they are quite 
different in their inception, derivation, and computation. The proof 
to follow will point out similarities between the process of counting 
agreements and inversions and the arithmetic manipulation of ranks. 


Proof that (P — Q)/(nom) = (2/n) (Y, — Yo), %.€., 79 = 1d 


It will be convenient to express (2/n) (Yı — Yo) in a different 
form. 


a aus fy ~ 2] 7,241] -2 [> saia As D]. 


NN, 
To establish the proof it is sufficient to show that 


adafa naeta] 


We note that P + Q = nyNo, hence P — Q = 2P — mn. 
Therefore, we shall prove that 


2P — mm =2 ¥, —m(n + 1), or 
2P = 2) Y, — m(n + 1) + mn, or (6) 
P= ŁY, - mmu + 11/2 
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Establishing (6) will establish that r, = rb. 
The following example will illustrate the proof: 


Sen Agreements Inversions 
6 3 

5 2 
4 2 
3 2 

2 

1 DEANA 

P=7 Q=2 


We shall show that a method of counting and summing agree- 
ments is equivalent to evaluating 3Y, — [m (m + 1)]/2. 

“Agreements” are defined only for those ranks which appear un- 
der column 1. We shall always arrange the ranks from highest to 
lowest. Suppose we wish to find the number of agreements associ- 
ated with the smallest rank under column 1, 3 in the above example. 
Denote this smallest rank under column 1 by Yi1. Now Yı, scores 
an agreement every time it exceeds a smaller rank under column 0. 
Obviously, Yı, will always exceed Yı, — 1 ranks which appear un- 
der column 0. How many agreements are associated with Ya, the 
next to the smallest rank under column 1 (4 in our example)? There 
are only Yo; — 1 ranks that are smaller than Y2, and all but one of 
these is in column 0 (Y. is in column 1); hence there are Yo; — 2 
agreements associated with Y2:. How many agreements are asso- 
ciated with Ys:? Of the Ys, — 1 ranks smaller than Yi, two of them, 
Yo, and Yı, are in column 1); hence, there are Ys: — 3 agreements 
associated with Y31. 

It should be easy to see that in general there are Yj, — j agree- 
ments associated with the jth smallest rank under column 1. The 
sum of these agreements is P, which we find as follows: 


E 2 TER 2 Y, — Iu(m + 1/2. 
This establishes the equivalence of r, and rb. 


Discussion 


It has been shown that a tau-type coefficient, which is not a cor- 
relation coefficient at all, derived by Cureton (1956), for use with 
a dichotomous and a ranking variable is algebraically equivalent to 
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a coefficient derived in this paper which estimates Spearman’s rho 
between the ranking variable and a hypothetical ranking vari- 
able assumed to underlie the dichotomy. The latter coefficient is 
clearly of the nature of a product-moment correlation coefficient. 
The equivalence holds only for the case of no ties in the observed 
ranking variable. When ties occur, rb and r, are no longer equiva- 
lent. 

These strange equivalences cropping up in various places in sta- 
tisties seem to indicate basic similarities between apparently dif- 
ferent concepts. For example, it is not widely known that Spear- 
man’s rho is the same simple function of weighted agreements and 
inversions that Kendall’s tau is of unweighted agreements and in- 
versions (Durbin and Stuart, 1951). 

Of what possible use is rb? (1) It measures monotonicity of re- 
lationship between the variables underlying Y and X’, (2) it al- 
ways lies on the interval —1 to +1, inclusive, (3) it is computa- 
tionally simple. These points are elaborated below: 


1. Unlike a correlation ratio which looks for any possible re- 
lationship between two variables and unlike Pearson’s r which 
measures linearity of relationship between two variables, rb 
measures an intermediate type of relationship between the vari- 
ables underlying the two sets of ranks. The value of rb is a 
measure of the monotonicity of relationship between the vari- 
ables underlying the two sets of ranks, Y and X’. Monotonicity 
is more general than linearity and less general than merely any 
sort of relationship. 

2. If we exclude those instances of no variance in either Y or 
X (which all correlation measures do), then it can be shown 
that rb has a minimum of —1 and a maximum of +1 regardless 
of the extremeness of the dichotomization on X or the nature 
of the variables underlying Y and X’. Researchers often experi- 
ence the unpleasantness of obtaining Pearson biserial correlation 
coefficients with values as high as 1.20. 

8. Formulas (2) and (3) above are particularly simple, and 
either (2) or (3) will be simpler depending on a given set of 
data. For untied ranks, formulas (2) and (3) give Cureton’s 
coefficient without listing data and counting inversions and 
agreements. 
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ESTIMATING GAINS IN SCHOLASTIC APTITUDE 
TEST SCORES ATTRIBUTABLE TO THREE SOURCES 


SAM C. WEBB 
Georgia Institute of Technology 


In recent years scores on the Scholastic Aptitude Test (SAT) 
have become increasingly important as measures of the intellectual 
quality of entering college freshman classes. Class profiles show- 
ing distributions of scores for applicants and for accepted and en- 
rolled students are routinely made available as a part of publicity 
and recruiting materials. Conversations concerning the quality of 
college classes tend to focus on average SAT scores as the princi- 
pal index of academic ability. Many institutions, desirous of im- 
proving their image in the academic world, point to gains in scores 
as evidence that admission standards have been raised. And 
changes in average scores have often been regarded as evidence 
for the increased effectiveness of recruiting and selection procedures. 

Though the high school average (HSA) or some derivative such 
as rank in class is for most colleges a more valid predictor of col- 
lege performance than are SAT scores, the former seems to receive 
less emphasis in these kinds of evaluations. There are understand- 
able reasons why this is so. 

SAT scores, for instance, are more objective than the HSA in that 
the SAT applies the same scale of measurement to all students re- 
gardless of the high school from which they come. High school 
averages, on the other hand, vary from school to school so that 
values of different schools are often not comparable; and class 
averages computed from records of different schools are in a tech- 
nical sense somewhat meaningless. More importantly, perhaps, the 
very nature of the grade scale as usually employed tends to ob- 
scure differences between grading standards among schools and 


633 


634 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


changes in standards within schools. The result is, of course, that 
the HSA is a rather poor measure for assessing changes in the qual- 
ity of admitted college classes. 

As already intimated, there have been in recent years gains in 
average SAT scores for entering freshman classes at a number of 
colleges. These gains usually have been interpreted as signifying a 
rise in the intellectual quality or grade-getting abilities of students. 
This sort of interpretation, however, has not received unanimous 
support. Some professors seem to feel that because of the rela- 
tively small increments in HSA averages and because of impres- 
sions they have over the years of grades made by students in their 
classes, these gains in average scores must be statistical artifacts 
emanating from the procedures employed in scaling the scores. 
However, since there are technical studies to show the scoring 
scales have remained quite stable over a considerable period of 
time (Angoff and Waite, 1959, 1961) and since there are empirical 
data that show practically no change in average SAT scores for all 
college applicants as reported by the College Entrance Examina- 
tion Board from 1957 to 1960 (Fishman, 1957, 1962) , this kind of rea- 
soning does not seem to provide an adequate explanation for the 
observed gains. 

Thus other explanations must be sought. Reasonably cogent and 
plausible explanations come to mind which suggest that the causes 
underlying reported gains may well be numerous and complex. 

These suggested causes may be grouped into two broad cate- 
gories: those that cannot be attributed directly, and perhaps not even 
indirectly, to the activity or influence of any particular college or 
colleges; and those that are essentially a function of the nature and 
activities of a college (or colleges) or closely associated individuals 
(or groups of persons). For want of better terminology these causes 
may be referred to as “non-college” and “college” causes. 

Several causes may be classified in the “non-college” group. 
These include a variety of factors which are psychological, educa- 
tional, sociological, psychometric, and perhaps even physiological 
in character. Some possibilities that readily come to mind are im- 
proved instruction and Preparation for college in high school, short 
term coaching effects—especially on the mathematical section for 
leh not taking math courses—increased test-taking ability or 
test “wiseness,” practice effects, and increased societal incentives 
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(psychological, sociological, and economic) which encourage in- 
creased numbers of able students to apply for admission to college. 
Probably of lesser importance but also worthy of inclusion here is 
the possibility of increased basic intellectual capacities of the popu- 
lation over time as a function of nutritional and physiological 
factors. 

The second or “college” factors category may be divided into two 
classes. In the first are included all those factors which are in- 
fluential in causing a student to apply for admission at a particular 
college. For convenience these are called “recruitment” factors. 
Suggested for inclusion in this class are various characteristics of 
the college—both actual and imagined; family associations and ties 
with the college; persuasive activities of friends, acquaintances, 
and peers; and the publicity and recruitment activities of the 
college. 

In the second class are included those factors which are ef- 
fective in the student’s enrollment in a particular college. Here are 
grouped all those factors which cause a student to be admitted by 
the college and which cause the student to enroll. For convenience 
these are called “admission and acceptance” factors. Associated 
with the selection process are the admissions procedures and poli- 
cies of the college and the characteristics of the student and his 
credentials and associations which are influential in his being se- 
lected for admission. Associated with the acceptance process are 
such factors as the scholarship or other financial resources available 
to the student through the college; the image of the college; and in- 
fluences of family, friends, acquaintances and friends already men- 
tioned. 

Except for the effects of short term coaching programs, there 
seems to be a scarcity of published studies on the effects of any of 
the suggested causative factors. This is regretable, for it would be 
both interesting and useful to assess what influence each of these 
suggested factors has in determining changes in average SAT 
scores. With such knowledge individual colleges could more mean- 
ingfully evaluate the effectiveness of their recruitment and selection 
procedures. And on a larger scale, perhaps, the relative importance 
of these factors as they affect students differentially among colleges 
could be investigated. 

Unfortunately, however, the very variety of possible causative 
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factors, and the probably high inter-relationships among some of 
them, leads one to suspect that rather complicated statistical and 
experimental designs and some rather expensive and long-term re- 
search would be required to secure satisfactory estimates of the 
effects they are producing. Consequently precise answers as to the 
effects of all these factors will probably not be forthcoming in the 
near future. 

In the meantime some approximate but nevertheless useful esti- 
mates of the effects of the three groups of factors already described 
—non-college, recruitment, and selection and acceptance—can be 
easily made when certain data are available. 

Three sets of average SAT scores, preferably broken down by 
test section and sex, are required. Each set should include averages 
for the beginning and ending of the same arbitrarily selected period 
of time. The first set should provide valid and unbiased estimates 
of the average scores for a base population comprised of high 
school students who will seek admission to college and who reside 
in a defined geographical area. Preferably this should be an area 
from which the college draws its students. The second set of data 
should provide average scores for the applicant population of the 
college; and the third set should provide average scores for appli- 
cants who enrolled as freshmen. 

From these data an estimate of the effects of ‘“non-college” 
causes on score changes over the period can be obtained by using 
averages for the base population and by subtracting averages for 
the beginning of the period from those for the end of the period. 
Estimates of the effects of the “recruitment” factors can be ob- 
tained in two stages. An estimate of the effects of these factors at 
the beginning of the period may be obtained by substracting aver- 
age scores for the base population at the beginning of the period 
from average scores for the college applicant population at the 
beginning of the period. Gain (or loss) in scores over the period re- 
sulting from recruitment factors may be obtained by first getting 
the difference between the applicant population and the base popu- 
lation average at the end of the period, and subtracting from this 
value the difference between the applicant population and base 
population average at the beginning of the period. 

In like fashion changes attributable to selection and acceptance 
factors can be estimated in two stages. Subtracting average scores 
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for students enrolled from averages for the college applicant popu- 
lation at the beginning of the period provides an estimate of se- 
lection effects obtaining at the beginning of the period. Gains re- 
sulting from these effects during the period may then be obtained 
by getting the difference between averages for enrolled students and 
the college applicant population at the end of the period and sub- 
tracting from this value gains existing at the beginning of the pe- 
riod already obtained. 

From these values total changes over the period resulting from 
all causes can be estimated. Tt is also possible to determine the total 
difference between the quality of students enrolled at the end of 
the period as contrasted with the regional population at the begin- 
ning of the period. 

The use of these procedures is illustrated with data for Emory 
College and for all predominately white public colleges of the state 
of Georgia (Klock, 1964) for the five year period covering the aca- 


demic years of 1958-59 through the year 1962-63. The SAT aver- 
ages employed are shown in Tables 1 and 2. In making estimates it 


TABLE 1 


Average SAT Scores for Applicant Populations of Emory College 
and Georgia White Public Colleges 


N L E a ee 


Academic 
Year 


1958-59 
1962-63 


Academic 
Year 


1958-59 
1962-63 


5 495 
(n = 717) (n= 586) 


Emory College Georgia Colleges 
oea S 
SAT-V SAT-M SAT-V SAT-M 


Ene E en O ESA TAT 
Male Female Male Female Male Female Male Fer 


AT 476 400 387 457 38 
(n = 4013) (n = 2068) 
519 528 558 527 1 1 472 4] 


(n = 1094) (n = 931) (n = 5796) (n = 3281) 


TABLE 2 
Average SAT Scores for Students Entering Emory College 
and Georgia White Public Colleges 


Emory College Georgia Colleges 
OCRE ee ee ee 
SAT-V SAT-M SAT-V SAT-N 


— MATE EESE tE LY 
Male Female Male Female Male Female Male Fei 


504 505 534 492 400 387 457 3 
(n = 275) (n = 195) (n = 4013) (n = 2068) 
548 570 589 575 439 432 494 4 


(n = 368) (n = 240) (n = 4201) (n = 2397) 
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has been assumed that the applicant population values for the 
Georgia system are valid and unbiased estimates of the base popu- 
lation of the region from which Emory College and the Georgia 
system draw their applicants. Associated with this assumption is 
the further one that recruitment effects for the system as a whole 
are of minimal consequence. It is further assumed that the data 
for the enrolled students for the Georgia public colleges in 1958-59 
are a valid estimate of the applicant population of the system for 
that year. This assumption seems justified on the basis of informa- 
tion which suggests that, except for the requirement of graduation 
from high school, there was practically no selectivity in admis- 
sions for the system as a whole in that year. 

The estimates for Emory College are given in Table 3; those for 
all Georgia white public colleges are given in Table 4. The large num- 
ber of exact zero entries in Table 4 appear because 1958-59 aver- 
ages for enrolled students of the Georgia system have been used as 
estimates for the base population and for the applicant popula- 
tion for the Georgia system for that year. 


TABLE 3 
An Analysis of Gains in SAT Scores for Emory College 
from 1958 through 1962 
SAT-V SAT-M 
A —— - Unweighted 

__Souros of Gain Male Female Male Female Average 
Non-College Factors +21 +34 15 2 
Recruitment Factors: Ki Ey 
Gain at beginning of 

D $ 75 108 52 90 81.3 
Gain at end of period 98 107 86 109 100.0 
Gain over period +23 -1 +34 +19 +18.7 
Selection and Accep- 

tance Factors: 
Gain at beginning of 

p > 29 10 25 16 
Gain at end of period 29 42 31 48 Ta 
Gain over period 0 +32 +6 +32 +17.5 
Total Gain over 5 

years 44 65 55 83 61.8 
Total Gain of 1962 

enrolled students 

over 1958 base ; 

population 148 183 182 189 163.0 


í 


ion Seo 


mS 
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TABLE 4 
An Analysis of Gains in SAT Scores for All Georgia White Public 
Colleges from 1958 through 1962 
SAT-V SAT-M 
Unweighted 

Sources of Gain Male Female Male Female Average 
Non-College Factors 421 +34 +15 +32 +25.5 
Recruitment Factors: 
Gain at beginning of 

period 0 0 0 0 0 
Gain at end of period 0 0 0 0 0 
Gain over period 0 0 0 0 0 
Selection and Accep- 

tance Factors: 
Gain at beginning of 

period 0 0 0 0 
Gain at end of period 18 11 22 9 15.0 
Gain over period +18 +11 +22 +9 +15.0 
Total Gain over 5 

years 39 45 37 41 40.5 
Total Gain of 1962 

enrolled students 

over 1958 base 

population 39 45 37 41 40.5 
pE. ee r A E ar n a 


For Emory College the data show unweighted average gains 
taken across tests and sexes during the period of about 26 points 
arising from non-college factors, of about 19 points from recruit- 
ment factors and of about 18 points from selection and acceptance 
factors. This yields an average gain from all causes of about 62 
points over the five year period. For white public colleges of Geor- 
gia there was the same 26 point gain from non-college factors and 
a 15 point gain from selection and acceptance factors. Thus there 
is about a 41 point gain over the period due to all factors. This 
means that there is about a 21 point greater gain for Emory than 
for the Georgia colleges. 

While this discussion has emphasized the analysis of gains within 
the five year period, it is clear from Tables 3 and 4 that the analy- 
sis also yields information concerning the magnitude of gains over 
the base population attributable to recruitment and to selection 
and acceptance factors existing at the beginning of the period. Thus 
for Emory College at the beginning of the period there were aver- 
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age gains of 81 and 20 points attributable to these two types of 
sources respectively. Further it can be noted that the average gain 
for the 1962 enrolled students over the 1958 base population was 
163 points for Emory College and 40 points for all Georgia white 
public colleges. 

For Emory College it is also interesting to note that while dif- 
ferences from recruitment factors in 1958-59 were larger for fe- 
males than for males, the gains over the five year period were larger 
for males than for females. Also while gains attributable to selec- 
tion and acceptance factors were greater for boys than for females 
at the beginning of the period, gains over the period were consider- 
ably larger for females. This latter difference is largely a function 
of limited housing facilities for women. In the aggregate, gains 
from both recruitment and selection and acceptance factors are 
larger for females than for males. 

In making estimates of the type described here, the greatest 
problem is that of finding averages that will be accepted as valid 
and unbiased estimates for an appropriately selected base popula- 
tion. In the illustrations used, for example, had data for the appli- 
cant and enrolled populations for Emory and the Georgia system 
for the years 1957-58 and 1960-61 been available, all applicants 
in the country taking the SAT for those years could have been 
used as the base population, since the College Entrance Examina- 
tion Board has published average scores for this group for those 
years. However, since Emory draws the majority of its students 
from the Southeast, and since the Georgia system draws its students 
primarily from the state of Georgia, a base population of college 
applicants in this region would seem more appropriate than one 
involving a larger geographical area, even though the estimates of 
averages may be less accurate. 

It should be noted that only three estimates are affected by the 
data for the base population. These are the estimates of “non- 
college” effects, and the two estimates of recruitment effects. And 
one may judge the effects that base population values considered 
to exceed or fall below some “true” value have on these estimates. 
For example, if one considers both base population values to be 
overestimated to the same degree, the estimate of gains due to re- 
cruitment factors at the beginning of the period will be underesti- 
mated. Then suppose one considers the base population values at 
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the beginning of the period to be underestimations and those at the 
end to be about right or over-estimations. Then gains due to non- 
college factors will be over-estimated; gains due to recruitment fac- 
tors at the beginning of the period will be over-estimated; and 
gains due to recruitment factors over the period will be under-esti- 
mated. 

In respect to the illustrative data used, several inquiries lead the 
writer to conclude that the data for the base populations are the 
best that are presently available for Georgia and the Southeastern 
region. If these values deviate from the “true” population values, 
they probably do so in the direction of being underestimates as a 
function of the abler students enrolling in private colleges or col- 
leges of other state systems. For economic reasons, these deviations 
are probably larger for the 1962-63 data than for the 1958-59 
data. If these speculations are correct, then gains attributable to 
non-college factors are underestimated and gains attributable to 
recruitment factors both at the beginning of the period and over 
the period are overestimated. 

In conclusion, it is to be emphasized that the procedures de- 
scribed provide a suggested way for estimating the contributions of 
three types of causes of changes in average SAT scores that occur 
over a period of time for enrolling freshman classes in a college or 
group of colleges. While the illustrative data involved only gains, 
the procedures seem general in application and could apply to any 
type of change. The results seem potentially useful in getting a rea- 
sonable evaluation of the influence of these groups of factors on 
changing average SAT scores for a college. 
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A STUDY OF SAMPLE SIZE IN MAKING DECISIONS 
ABOUT INSTRUCTIONAL MATERIALS? 


LAWRENCE M. STOLUROW 
Harvard University 
AND 
GERALD FRINCKE 
Sacramento State College 


Tun present study was conducted to explicate the problems in- 
volved in making decisions about the acceptability of frames in 
self-instructional programs. Its approach, different from that of 
Holland (1965) using the “blacked-out” procedure to determine 
the effects of stimulus degradation on error rate and that suggested 
by Stolurow (1965) using the redundancy test. In the present 
study, data were examined to determine the relative efficiency of 
various sample sizes in making decisions about retaining or re- 
vising the frames of a self-instructional program. Individual frames 
are not assumed to be statistically independent of one another. The 
errors made on one frame are likely to be correlated with those 
made on another and the magnitude of the correlations is generally 
both unknown and variable throughout a program. This state of 
affairs indicates the need for an empirical study of sampling prob- 
lems in decision making. Two indices were used to determine the 
implications of selection criteria and sampling procedures in de- 
ciding about the acceptability of frames. One index was the per 


1 This paper is based upon research supported, in part, by the U. S. Office 
of Education, Educational Media Branch, Title VII, Grant No. 7-23-1020- 
151.1. Co-investigators were Lawrence M. Stolurow and Max Beberman. This 
study was issued originally as Technical Report No. 11, February, 1965, by the 
Training Research Laboratory, University of Illinois, under the same authorship 
and title. Reproduction in whole or in part is permitted for any purpose of the 
United States Government. 
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cent of undesirable frames correctly identified. The other was the 
per cent of rejections made erroneously (Type I errors). The study 
also was conducted to suggest guidelines for determining the sample 
size to use in developing self-instructional programs in which it is 
expected that the overall error rates will be low and the distribu- 
tion of error rates observed for frames within the program will be 
skewed. 


Decision Making and Types of Errors 


In the course of evaluating and revising programed instructional 
materials, a programer may decide to reject as undesirable those 
frames in the program which he suspects will lead to a student error 
tate above a given criterion value. In doing this, the programer is 
in a position similar to that of a statistician faced with a large 
number of hypotheses to test. Each frame in the program must be 
accepted or rejected. In effect, for each frame, the programer must 
test the null hypothesis that the frame will have an error rate below 
that value he has chosen as his minimum criterion for “undesir- 
ableness” when the program is put into general use. The number 
of observations on which each test of this hypothesis is based will 
be equal to the number of students with whom the program is pre- 
tested. Usually (though not necessarily) the programer rejects the 
null hypothesis whenever a frame is found to have an error rate 
equal to or greater than the one chosen as the criterion value. 

In making each decision concerning the acceptability of a pro- 
gram frame, the programer, as statistician, may make one of two 
types of errors. He will make 
is rejected when it is true; he 
“unacceptable.” i 


able frame as “acceptable. 
Type II errors. He wan 
test of the null hypothe 
each truly unacceptable 
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Power, Sample Size and Student Ability 


Given a fixed number of observations (N), the only way to 
change the probability of rejecting an acceptable frame is to shift 
the rejection criterion. Unfortunately, in this situation a shift of 
the rejection criterion which reduces chances of this Type I error 
simultaneously reduces the power of the test. Conversely, a shift of 
the criterion which increases the power of the test results in the in- 
creased probability of a Type I error. Only by increasing N can the 
power of the test be increased without a simultaneous increase in 
the probability of a Type I error. Similarly, only by increasing N 
can the probability of a Type I error be reduced without diminish- 
ing the power of the test. 

Since each test is based on the error data of students utilized in 
pretesting the program, the power of each test will be inversely re- 
lated to the performance level or overall error rate of the N stu- 
dents comprising the sample. If the N students perform quite well, 
error rate estimates will be depressed for all items. Consequently, 
fewer unacceptable items will be rejected. Conversely, if the N stu- 
dents perform quite poorly, observed error rates will be higher and 
more frames (both acceptable and not) will be rejected. This 
suggests the desirability of obtaining measures characterizing the 
sample of students used in terms of relevant abilities, and of ob- 
taining measures of their representativeness in terms of relevant 
academic achievement. 


True Error Rate and Rejection 


We can speak of the “true” error rate of a frame as that error 
rate which would be found if the frame were given to all of the 
intended population as a part of the finished program. The power 
of each test made by the programer also depends upon the degree 
to which the actual state of affairs approaches that stated in the 
null hypothesis. Thus “unacceptable” frames with very high true 
error rates are more likely to be rejected on the basis of pretesting 
data than unacceptable frames with true error rates which more 
closely approach acceptability. The programmer may thus expect 
that the percentage of truly unacceptable frames rejected by tests 
based on data from a given pretesting sample of size N will not 
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be very high in the case of programs which have already under- 
gone fairly successful revision. 


Method 
Materials 


The program. The University of Illinois Committee on School 
Mathematics, (UICSM) programed instructional series, Part 110, 
was selected for study since it teaches relatively difficult mathe- 
matical concepts and thus some variability in error rates for frames 
could be expected. Worksheet error data for 178 of the students 
who completed (pure mode) Part 110 (Beberman and Stolurow, 
1963) were recorded on SCRIBE? sheets and used with the 
SCRIBE system to produce IBM cards? containing individual er- 
ror data for each student. Error data from the completed work- 
sheets of the 52 additional pure-mode subjects were punched di- 
rectly into IBM cards in the SCRIBE output format. Thus, data 
from 230 students were available for sampling, 

Sampling procedure. All sampling of data from the basic pool of 
230 students was done without replacement. A stratified random 
sample of 100 students from all seven classes in four participating 
schools was selected to establish criterion error rate measures on 
the 308 program frames in UICSM PIP Part 110. A similar strati- 
fied random sampling procedure was then followed as closely as 
possible in selecting several other samples of various sizes. In 
cases where a stratified procedure was clearly impossible, such as 
in a sample of size 1 (N = 1), ordinary random sampling was uti- 
lized. Three samples each of the following sizes were selected: N — 
1,N=2,N=3,N = 4, N =5,N = 10, and N = 15. These 


samples were also merged to form a “summation sample” with N = 
120. 


2 SCRIBE is a system developed and used by ETS 


answer sheets and to transcrib 


to score multiple-choice 
to IBM cards. pay 


the data on the answer sheets 


processing worksheet data is d 
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Item analyses.‘ Worksheet error data cards prepared from the 
worksheets of the subjects who constituted the criterion sample, the 
summation sample, and the 21 smaller samples, were used as the 
basis for 23 separate item-analyses of the 308 items which com- 
prise the UICSM Programed Instruction Part 110. 

Three hundred and eight summary IBM cards were then pre- 
pared. Each of these cards contained the results of all 23 analyses 
with regard to one of the 308 program items. The summary cards 
were used to determine correlations between the results of item 
analyses based on the various samples. These cards were also used 
to determine which frames would be rejected and which accepted 
by tests based on the data of the various samples in cases where the 
criterion for rejection would be the observed error rates equal to 
or greater than 10%, 15%, or 20%. 


Results 


Figure 1 is a frequency distribution of the different error rates 
observed in Part 110 for the criterion sample (N = 100). Table 1 


“a 


40 


5% 10% s% 20% 25% 30% 3% 40% 45% 
Figure 1. Frequency distribution of error rates as determined with the criterion 
sample (N = 100). 


4 These analyses were carried out with the aid of an IBM 1620 computer 
and a program written by Scott Krueger, University of Illinois, Training Re- 
search Laboratory. 
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shows the distribution of error rates observed in the 21 smaller 
samples. All of the distributions are quite skewed. Most items are 
well within the limits of acceptability and the number of extremely 
unacceptable items in Part 110 is actually quite low. This is an 
important factor in interpreting the findings of this study. 


TABLE 1 
Distribution Errors in Each in 21 Small Samples* 


Number of students who made errors 
Sample 


Total 
sie Cth ees a) 8 6.7 8-15, frames 
1 286 22 308 
1 300 08 308 
1 296 12 308 
2 20. #45 2 308 
2 26 4 1 308 
2 37. O ET 308 
3 DNS) E LT 308 
3 m1 86820—~«“itCO 308 
Soman 287 CBO 1) 40 308 
4 28 15 5 0 0 308 
4 Oe 78. Wk 2 0 308 
4 AREON seu iL) VT 308 
5 ce e oa NO 308 
5 2 6 6 4 0 O 308 
5 SAME Oley RB Kd, :e\0x 0:0 308 
10 ORAT DS E hes tig. ON. O 308 
10 AERA ASEA SY 4G!) ar Oh kOe 0 308 
10 AGOCMIE C7) EASO ICD! KAW LON: ROMUMOM HO 308 
15 ATU (ioe) 57 SAMO AAS Uinta (ean Tee MAY 308 
15 LOTET nS) ESANEAN I O 308 
15 NO S EAA E AS OED O a depo 308 


* Based on UIOSM Programed Intruction Booklet Part 110. 


Table 2 presents correlations between the error rates of the 308 
items as determined by each of the samples and error rates based 
on the criterion sample. These correlations are generally quite low 
as might be expected due to the small range of error rates obtained 
and the extreme skewness of the error rate distributions. 

Estimates of the overall program error rate based on the 21 
independent samples also are presented in Table 2. Considerable 
variability among the estimates based on the smaller samples can 
be seen. For example, those for samples of size 1 range from 2.5 per 
cent to 7.1 per cent, those of size 5 range from 4.8 per cent to 11.7 

per cent overall error rate. Figure 2 shows the mean overall error 
rate estimates for each of the sample sizes in relation to the over- 
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TABLE 2 


Correlations of Sample Item Analyses with Criterion Item 
Analyses as a Function of Sample Size 


Sample Estimate of 
program Correlation 
Code error rate with 
No. Size in% criterion® 

0 100 5.6 1.000 
1 1 TE .329 
2 1 2.5 .288 
3 1 3.6 -101 
4 2 8.0 356 
5 2 6.9 -297 
6 2 3.6 318 
7 3 5.0 .272 
8 3 4.5 .352 
9 3 5.6 .302 
10 4 2.0 .455 
11 4 8.7 .481 
12 4 8.4 .462 
13 5 11.7 .453 
14 5 7.2 .470 
15 5 4.8 .535 
16 10 3.9 .567 
17 10 8.0 .514 
18 10 5.5 .552 
19 15 3.7 .680 
20 15 4.5 -710 
21 15 5.1 641 
2% 120 5.5 .872 


a Criterion sample. 
» Summation sample. 
o Analysis is based upon 308 frames of UICSM PIP Part 110. All correlations except .101 are 


significant beyond the .01 level. 
all error rate observed in the criterion sample. All of these over- 
all rates appear to fluctuate randomly about the criterion value 
with the exception of that based on the samples of N = 5. Here 
a very excessive and presumably atypical error rate was observed. 
Figure 3 depicts the efficiency of the seven sample sizes by show- 
ing the relationship between the mean percentage of unacceptable 
frames that were rejected and sample size. It shows that all sample 
sizes greater than N = 4, with the exception of the sample size 
N = 10 at the 15 per cent criterion level, rejected 50 per cent or 
more of the unacceptable items. In all cases, a rapid rise in the 
mean percentages of the unacceptable items correctly identified for 
rejection can be seen in relation to the sample size as it increases 
from one to five. Each line represents a different criterion. In the 
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10 6 20 
Sample Size 


Figure 2. Mean overall error rate estimates as a function of sample size. 


case of a 10 per cent error rate criterion, the efficiency of the sample 
size increases up to size 10. With a 15 per cent or 20 per cent error 
rate, the efficiency of the sample in terms of the rejection of un- 


[i 2 3 45 


© 5 120 
Figure 3. Percentage of the unacceptable items rejected asa function of sample 
size and rejection criterion. 
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acceptable items at first increases up to sample size 5 even more 
rapidly than the curve for the 10 per cent criterion. However, these 
curves decrease as sample size is increased beyond N = 5 and then 
increased as sample size goes from 15 to 120. Table 3 presents both 


TABLE 3 
Percentage and Number of Unacceptable Frames that Would be Rejected 
‘as a Function of the Size of Sample and the Criterion 
Error Rate Used for Rejection 


Criterion error rate used for rejection 


E E A AEE eS Se 
Sample 20% 15%° 10% 
Size No. % No. % No. % No. 
100 0s 100 IE 100 24 100 63» 
1 il 45 5 29 7 19 «12 
1 2 18 2 17 4 10 6 
1 3 18 2 8 2 5 3 
2 4 45 5 46 il 38 24 
2 5 55 6 38 9 30 19 
2 6 36 4 21 5 22 14 
3 7 36 4 29 7 29 18 
3 8 64 7 42 10 32 20 
3. (9 36 4 54 13 35 22 
4 10 45 5 50 12 25 16 
4 i 64 7 737 19 62 39 
4 12 73 8 67 16 63 40 
5 13 100 11 92 22 68 43 
5 14 91 10 58 14 57) 36 
5 15 100 11l 88 21 52 33 
10 16 74 7 42 10 5635 
10 17 45 5 38 9 78 49 
10 18 64 7 62 15 70 44 
15 19 45 5 29 7 51 32 
15 20 82 9 58 14 51 32 
15 21 45 5 92 2 52 33 
120 22 73 8 62 15 68 43 

(Summation Sample) 
a Criterion sample. 


b11 frames were unacceptable at this criterion level. 
e 24 frames were unacceptable at this criterion level. 
4 63 frames were unacceptable at this criterion level. 


the percentage and number of unacceptable items correctly identified 
by each of the samples of subjects. It should be noted that for a 10 
per cent criterion, for example, all percentages are above 50 only 
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for samples of N = 5 or more. However, with a 15 per cent criterion 
not all the samples of size 15 resulted in 50 per cent or greater correct 
identifications of unacceptable frames. 

Table 4 shows the number and percentage of the acceptable items 


TABLE 4 


Percentage and Number of Acceptable Frames that Would be Erroneously 
Rejected as a Function of the Size of Sample and the Criterion 


Error Rate Used for Rejection 
Criterion error rate used for rejection 
Sample 20%» 15% 10%! 
Size No. % No. % No. % No. 
100 Os 0 0 0 0 0 0 
1 1 5.7 17 5.3 15 4.1 10 
1 2 2.0 6 1.4 4 0.8 2 
1 3 3.4 10 3.5 10 3.7 9 
2 4 14.1 42 12.7 36 9.4 23 
2 5 12.1 36 11.6 33 9.4 23 
2 6 5.7 17 5.3 15 2.9 7 
3 7 12.5 37 12.0 34 9.4 23 
3 8 10.1 30 9.5 27 6.9 17 
3 9 15.8 47 13.4 38 11 8 29 
4 10 5.1 15 2.8 8 1.6 4 
4 11 28.3 84 25.7 73 21.6 53 
4 12 27.6 82 26.1 74 20.4 50 
5 13 40.4 120 38.4 109 35.9 88 
5 14 26.3 78 26.1 74 21.2 52 
5 15 18.9 56 16.2 46 13.9 34 
10 16 6.7 20 6.0 17 19.6 48 
10 17 17.2 51 16.5 47 43.7 107 
10 18 10.8 32 8.5 24 30.2 74 
15 19 2.4 7 1.8 5 2.4 6 
1520 i nh ATEN, 78 61 15 
15 21 2.7 8 1.8 5 11.0 27 
120 22 1.0 3 
(Summation Sample) R H si A 
* Criterion sample. 


» 207 frames were acceptable at this criterion level 
‘284 frames were acceptable at this criterion level, 
4245 frames were acceptable at this criterion level 


that would be erroneously rejected by each of the samples using the 
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w. (85.3 4 5 10 15 120 
Sample Size 


Figure 4. Per cent of rejections made erroneously as a function of sample size 
and rejection criterion. 


in terms of their percentage of all rejections made and thus serves 
to indicate the “cost” of each correct rejection in terms of incor- 
rectly rejected items. 


Discussion 
Skewness Effects 


The extreme positive skewness of the error rate distribution ap- 
pears to affect the efficiency of a given sample size in leading to the 
rejection of unacceptable frames. It seems to have lowered the ef- 
ficiency of N when the rejection criterion was reduced from a 20 
per cent error rate to a 10 per cent error rate. A shift in the rejection 
criterion of this sort moves the point of rejection to a place in the 
frequency distribution where considerably more frames are accu- 
mulated, since the frames, in general, tend to produce small numbers 
of errors. 

Since the power of the test is lowest for frames which have true 
error rates that are almost acceptable, the overall efficiency of the 
tests of frame acceptability is lowered in this situation. One factor, 
however, works to counteract this reduction in efficiency to some 
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extent when N is small. It is clear from the formula for the standard 
error of a proportion op = \/p(1 — p)/N that as the proportion 
(p) is shifted away from .5, the standard error is reduced. Thus, 
the standard error of the error rates (proportion of errors) which 
are observed for borderline unacceptable items will be reduced when 
the point of rejection is shifted further away from the 50 per cent 
error rate value. As can be seen from the formula, this effect be- 
comes less important as N is increased. 


Overall Error Rates 


The mean overall error rates observed for the various sample 
sizes as depicted in Figure 2 must be considered when interpreting 
the efficiency curves in Figure 3. Of major concern here is the fact 
that the mean overall error rate of the samples with N = 5 was 
considerably above that of the criterion. Thus, the efficiency curves 
in Figure 3 are higher for N = 5 than would normally be expected. 
The same curves are somewhat depressed at the point where N = 


15 due to the fact that the mean overall error rate happened to be 
lowest for samples of this size. 


Detection of Unacceptable Frames 


Two major relationships are illustrated in Figure 3. The first is 
that in detecting unacceptable frames, samples with N = 5 or 
N = 10 can approach quite closely the efficiency of much larger 
samples. Inspection of Table 8, however, shows that while the 
mean efficiency of the smaller samples is high, variability in ef- 
ficiency is also quite high. Thus, the programer may be able to ap- 
proach the efficiency of a very large sample with only a small one, 
but he runs a definite risk of drawing a very poor sample. It should 
be noted that even the summation sample leaves a great deal to be 
desired in the identification of faulty frames. 

While it is possible to identify three-fourths or more of the un- 


acceptable frames with samples as small as 10 when a 10 per cent 
criterion is used, samplin, 


j g fluctuations are great enough that, in 
this study, no sample of size 15 was this efficient. The data indicate 
that when a higher error rate is used as a criterion, smaller sam- 
ples may identify three-fourths, or more, of the unacceptable frames, 
and that this could occur with samples as small as 4. While it may 
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not occur with even larger samples, the chances are greater that it 
will. 


Sample Size and Rejection Criteria 


A second relationship, shown in Figure 3, involves sample size and 
the rejection criterion. The curve for the 10 per cent criterion reaches 
its first maximum when N = 10, while the curve for the 20 per cent 
criterion reaches its first maximum at N = 5. This is due to the 
fact that with these Ns and these criteria, N is as large as possible 
at the points where one subject missing a frame will cause it to be 
rejected. In other words, the lowest possible error rate other than 
0 per cent which can be observed in these samples is equal to the 
rejection criterion at these points. This is not true for samples with 
slightly smaller or slightly larger Ns. For example, when the crite- 
rion for rejection was a 20 per cent error rate and N = 3, the only 
possible observed error rates were 0%, 33%, 67% and 100%. Thus, 
if a frame was to be rejected, 33 per cent or more of the students 

` had to miss it. This means, in effect, that actions taken in making 
a decision to accept or reject a frame were the same as if the rejec- 
tion criterion was shifted to an observed error rate of 33 per cent, 
while the null hypothesis being tested continued to be that the 
frame has an error rate less than 20 per cent. The effect of this 
de facto rejection criterion shift is a reduction in the power of the 
test of this hypothesis. The probability of erroneous rejections (Type 
I errors) would, however, be reduced. When N = 5 in this case, 
possible observed error rates were 0%, 20%, 40%, 60%, 80% and 
100%. Thus, no de facto shift in the rejection criterion occurred, 
since an observed error rate of 20 per cent led to rejection of the 
item. Increasing N from N = 5 to N = 6 would, according to the 
same principle, reduce the power of the test. With N = 6, possible 
observed error rates are 0%, 17%, 33%, 50%, 67%, 83% and 100%. 
Again, a de facto shift of the rejection criterion to 33 per cent would 
_ occur and the power of the test would be reduced. 
According to this analysis, it would be expected that if more 
-sample sizes had been included in the present study, then each of 
the curves in Figure 3 would regularly rise and fall as N increased 
from 0 to 120. Each successive maximum would be a little higher 
than the last due to increased power brought about by increasing 
N. Each successive minimum would also be higher due to closer ap- 
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proximation of the de facto rejection criterion to the desired re- 
jection criterion. The maxima would always be at points where the 
product of the criterion per cent and N is a whole number. For some 
criteria the maxima will occur with N equal to an integer. For 
others, N will not always be an integer but will sometimes be an 
integer plus a fraction. In each of these cases, the observed maxi- 
mum would be the integer with the fraction dropped. This would 
be the case for a criterion of 15 per cent. The first maximum for this 
criterion would theoretically be with N = 6.7. The observable 
maximum would be at N = 6, however, since a sample with N = 
6.7 cannot be obtained. 


Efficiency in Terms of Type I Errors 


Thus far, the discussion has concerned the efficiency of various 
sample sizes in terms of the percentage of unacceptable frames re- 
jected. Efficiency should also be evaluated in terms of the extent of 
Type I errors, the rejection of acceptable frames. Examination of 
Table 4 reveals that large numbers of acceptable frames were re- 
jected by tests based on the smaller samples. When these numbers 
are presented as per cents of all rejections made, as in Figure 4, the 
lower efficiency of smaller sample sizes becomes quite clear. For 
samples with N less than 15, the best samples led to the rejection of 
at least one acceptable frame along with every unacceptable frame 
rejected. The worst rejected as many as nine acceptable frames for 
every unacceptable frame rejected. The curves in Figure 4 as in 
Figure 3 have probably been influenced by the mean overall error 
rates observed with the various sample sizes. Thus, they are some- 
what higher where N = 5 and somewhat lower at N = 15 than 
would normally be expected. Figure 4 definitely shows, however, 
that one advantage of using a large pretesting sample is that erron- 
eous rejections are considerably reduced. 
ate . ee that the percentage of the rejections made er- 
meee Ae e meets of the rejection criterion. For smaller 
= 4 ast, tl e percentage of rejections made erroneously is 
2 S when the rejection criterion is reduced from a 20 per cent 
wang Ae ea Such a shift in the rejection criterion 
ssa ae pe a least three things which would affect the 
pier Oraz jections made erroneously. First, by lowering the 

Jection criterion, the proportion of acceptable frames in the pro- 


i 
= 
a 
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gram is reduced and the proportion of unacceptable frames in- 
creased. This reduces the probability that an error made by a stu- 
dent will be made on an acceptable frame. Thus, fewer acceptable 
frames are rejected. Second, the lowering of the criterion places 
more frames in the “almost unacceptable” category due to the 
skewness of the error rate distribution. This would tend to increase 
erroneous rejections. The power of the test in making proper rejec- 
tions would also be reduced, since more “almost unacceptable” 
frames would be close to the criterion point. The effect of these 
changes would be to increase the per cent of rejections made er- 
roneously. A third result of a downward criterion shift would be to 
reduce the mean standard error of the acceptable frames for rea- 
sons already mentioned. Such a reduction in variability lowers the 
percentage of erroneous rejections. With a large N, however, this 
effect is considerably smaller. The net effect of all these changes 
depends considerably on the distribution of error rates in the pro- 
gram. 


Hazards of Small Samples 


The wide variation in efficiency among samples of a given size, 
in terms of rejecting unacceptable frames and failing to reject ac- 
ceptable ones, also obtains where the prediction of overall pro- 
gram error rate is concerned. This is readily seen when inspecting 
the overall error rate estimates presented in Table 2. 

The failure to obtain consistent results with smaller samples in 
the present study points up a major objection to the use of small 
pretesting samples. One cannot be confident that a small sample 
of students will produce individual and overall error rates con- 
sistent with those which would obtain in the population for which 
the program was intended. This objection, along with the fact that 
erroneous rejections are quite frequent when small samples are 
employed, must be seriously considered by the planner of a pre- 
testing program. The cost of failing to reject unacceptable frames, 
of rejecting acceptable frames, and of inaccurately estimating the 
overall error rate for a program must be balanced against the cost 
of pretesting the program. When these things are considered the N 
of the pretesting sample should be set as large as is practical. It 
should be chosen so that the product of the desired rejection cri- 
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terion and N is an integer. This will maximize the power of the 
test. 


Sequential Testing 


In cases where the program can be pretested with a very large 
number of students, the programer may wish to adopt a sequential 
testing procedure. He could use a sample of moderate size and a 
very strict rejection criterion for the first test. According to our 
findings, this should identify most of the unacceptable frames al- 
though several acceptable frames would be erroneously rejected. 
The programer would then restrict his frame analyses based on the 
remainder of the pretesting data to those frames rejected by the 
first test. Here the normal rejection criterion would be employed 
with a large N. This second test would reduce the probability of 
erroneous rejections and simultaneously eliminate the necessity of 
carrying out large-scale item analyses for the majority of the pro- 
gram frames since most of these frames will have error rates con- 
siderably below the rejection criterion. 


Summary 


In spite of the common practice, in developing programed learn- 
ing materials, of using small samples of students from the target 
population to accept or reject frames, there has been no examina- 
tion of the implications of this practice. This study relates the 
problem to the problem of the statistician who is testing a large 
number of hypotheses. The concepts of rejection level, Type I and 
TI errors, and the statistical concept of the power of a test are ap- 
plied. The empirical nature of the study is important since it is 
characteristic of the errors made to be intercorrelated and to form 
a skewed distribution with a mean that departs substantially from 
‘5. Twenty-one independent samples of seven different sizes and 
three per size were drawn from student worksheets used in 
learning from an algebra program based upon the UICSM cur- 
riculum. The hazards of small samples (up to N = 15) with re- 
jection criterion levels of 10%, 15% and 20% were examined. Wide 
Variations in efficiency among samples of a given size were ob- 
served both in terms of (a) rejection of acceptable frames, and (b) 
failure to reject unacceptable ones. Coupled with the iteng 
of small pretesting samples is the high frequency of erroneous re- 
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jections. It was recommended that pretest samples be both as large 
as practical and chosen, so that the product of the desired rejection 
criterion and N are integers, so as to maximize the power of the test. 
In some cases a sequential sampling procedure may be advisable. 
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THE REGRESSION TOWARD THE MEAN IN MMPI, 
CALIFORNIA PSYCHOLOGICAL INVENTORY AND 
SYMPTOM CHECK LIST 


R. M, JURJEVICH 
NP Clinic, 3415 Dispensary, Lowry AF Base, Denver, Colorado 


Tux problem of regression toward the mean of retest scores ap- 
pears insufficiently explored. It has been demonstrated with dif- 
ferent instruments and populations. On the other hand some test 
scales and some populations do not show the regression effect (cf. 
Dahlstrom and Welsh, 1960, p. 384). Some investigators also seem 
to take it for granted that the regression toward the mean would be 
more marked with high initial scores than the low ones. The sup- 
position is logical as there is more latitude for changes in extreme 
scores than in the moderately elevated ones. However, the statisti- 
cal verification does not bear out that supposition, as a general rule, 
for the 45 scales investigated. 


Method 


In studies preliminary to evaluation of individual and group 
psychotherapy programs (Jurjevich, 1966a, 1966b), the AF male 
personnel were retested after a short interval (SI) and long inter- 
val (LI). The mean interval for the SI group was about 10-12 days, 
and for the LI group about 180-210 days. The SI group had 51 Ss 
with MMPI retests, 43 with California Psychological Inventory 
(CPI), and 52 for Symptom Check List (SCL). The corresponding 
number in the LI group were: 55, 50, 56. The range of initial scores 
for each scale in the SI and LI groups was divided at the midpoint 
into high scores (HS) in the higher two quartals and low scores 
(LS) in the lower two. The differences between the test and retest 
scores in HS and LS were calculated on the ¢ tests. 
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Results and Discussion 


Table 1 shows that among the 45 scales employed, significant dif - 
ferences in amounts of regression, d, were found on seven scales in 
the SI group, 25 in the LI, and 23 in the combined SI + LI group. 
When the criterion of significance is widened to include scales sig- 
nificantly different at the level of p = .06 — .10, the corresponding 
number of significant differences becomes 10, 28, and 25 in the SI, 
LI, and SI + LI groups respectively. Apparently the regression 
toward the mean is more pronounced on some scales while being 
insignificant on others. The regression towards the mean is also 
more marked after a longer interval between test and retest than 
after a short interval, although this does not hold true in all cases. 
There are 10 scales (MMPI: Pd, Mf, At, Hv; CPI: Do, Re, Sc, 
Cm, Py; SCL raw?) which show no differences in regression toward 
the mean in any of the three conditions (SI, LI, and SI+-LI). Some 
scales exhibit significantly different regression effects in all three 
conditions (MMPI: F, Pa; CPI: Sp, Fz). Two scales reach signifi- 
cant differences, on the SI only (MMPI: Cn, Jh), six on the LI 
only (MMPI: L, K, Ma, Hc; CPI: To, Gi), and four in the SI + 
LI only (MMPI: Ho, Pd + 4K; CPI: Wb; SCL weighted). Six- 
teen scales show differential regression effects on LI and SI + LI, 
but not SI (MMPI: Hs, D, Hy, Pt, Sc, Si, R, Hs + 6K, Pt + K, 
Sc + K, Ma + .2K; CPI: Ie, Cs, Sa, Ac, Ai). Two of the scales 
(CPI: So, Fe) have significant differences in the SI and SI + LI, 
but not the LI, and one scale (CPI: Sy) in the LI and SI, but not 
the SI + LI. It is evident that the regression effects depend largely 
on the individual characteristics of a scale, so that no general rule as 
to regression effects can be postulated for the self-assessment scales 
employed in this study. 

The number of Ss falling into HS and LS categories is relatively 
even on some scales and uneven on others. In some cases these un- 
even Ns may act as suppressors of the significant regression effects, 
as the calculations for ¢ were done by Walker and Ley (1953) 
formula 7.23. If it be considered that a difference of 51 per cent or 
more between Ns of the HS and LS group might be large enough to 
influence the level of significance on the ¢ test, it is found that there 

1 The “SCL raw” represents the number 


. eto of symptoms admitted, the “SCL 
weighted” indicates the degree of trouble experienced. : 


R. M. JURJEVICH 
TABLE 1 


in the Short (SI) and Long (LI) Test-retest Intervals 
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The Differences of Means (d) of the High (HS) and Low Scores (LS) on Initial Tests 


dof HS — LS dof HS — LS 
Scale SI LI SI+LI Scale SI LI SI+LI 
MMPI raw CPI raw 
L 73 h h Y ho .95* Do .38 2.54 1.03 
Lie Dominance 
F 3.34" 6.29** 4.62** Cs .55 3.90** 2.21* 
| Validity Capacity for 
K —.30 1.90** .45 status 
Correction Sy 2.67* 3.89* 3,31 
Hs 42 5.76** 3.48** Sociability 
Hypochondriasis Sp 2.69% 6.71** 4.92** 
D —1.32 4.99* 2.048 . Social presence 
Depression Sa 1:99 2)70* .2.31** 
ually „51 4.06** 2,35** Self-acceptance 
Hysteria Wb 43 3.43 4.01** 
Pd —1.47 1.59 Ag Well being 
Psychopathic Re —.58 1.59 -68 
| deviate Responsibility 
Mf .30 «95 58 So —3.51** —.06 1.878 
Masculinity Socialization 
femininity Se —.66 2.36 1.12 
Pa 1.87" 4.80** 3.26** Self-control 
Paranoia To -75 4.08* 2.39 
Pt 1.93 7.02** 4.56** Tolerance 
Psychasthenia Gi 16  3.65° -1.74 
Se .13 15.58** 6.96** Good impression 
Schizophrenia Cm 67 2.64 1.01 
Ma AT 2.21* 1.37 Communality 
Hypomania Ac 2.25 6.34"* 4.37%" 
Si .00 4.628 3.07* Achievement via 
Introversion conformance 
‘yt -1.34 1.44 —.20 Ai 46  3.72** = 2.24"* 
‘Anxiety, Taylor Achievement via 
Cn 3.30* —.14 1.49 independence 
Control Ie .30  5.97* 3.49* 
He —.17 3.28* 1.59 Intellectual 
Hostility control efficiency 
_ Ho —2.07** 4.27* 1.16 Py 1.17 3,63 2.53 
_ Hostility, Cook Psychological 
Ho -86 “17 mindedness 
Overt hostility Fr 2.47** 2.89* 2.91** 
Jh —2.15* Al —.75 Flexibility 
_ Judged hostility Fe 1.24 1.56 1.35* 
R 1.18 2.94* 2.01** Femininity 
Repression Symptom CHECK LIST 
MMPI K corrected raw Ha 3.18 2.34 
Hs + 5K —1.44 6.09% 7.41** weighted 1.60 19.24 9.43* 
Pd + 4K 2.26 5.65 4.73* 
aPt +K ESZ 15:73%* 7.10% 
‘So +K 3.39 18.34** 12.16** 
Ma + 2K 1.73 6.11* 4.07* 


*#lt = .05 significant difference. 
** £ = .01 significant difference. 


"t] = .06-.10 significant,difference. 


` 
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are 39 scales having disparities in Ns that large. Twenty three of 
these scales show a significant difference between means of the HS 
and LS groups in spite of the suppressing effect of uneven Ns. The 
remaining sixteen scales represent less than 12 per cent of the total 
number investigated, indicating that disparities in Ns did not con- 
tribute substantially to the results obtained. 

In conclusion it may be pointed out that if the scales with sig- 
nificant regression toward the mean were used in assessing per- 
sonality changes, it would be essential to equate the groups on ini- 
tial scores to avoid the contaminating effects of regression. Such a 
procedure does not appear important for scales not showing dif- 
ferent regression effects of the HS and LS. 


Summary 


Groups of about 50 AF males were retested after intervals of 
‘about 10 and 180 days. These groups were divided into low and 
high scorers and differences in regression toward the mean de- 
termined. More than half of 45 scales show no different regression 
with high and low initial scores. Other scales show individualized 


patterns, depending partly on the length of interval between test 
and retest. 
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THREE EQUIVALENT FORMS OF A SEMANTIC 
DIFFERENTIAL INVENTORY! 


LOLAFAYE COYNE anp PHILIP 8. HOLZMAN 
The Menninger Foundation 


In the course of studying people’s reactions to hearing their own 
voices, we constructed three equivalent forms of a semantic dif- 
ferential inventory. We required that the forms be short enough to 
be filled out in less than one minute, yet long enough to yield stable 
scores. We used the forms to trace the momentary attitude changes 
towards one’s own voice at three successive but closely spaced 
times. Our experimental purpose, that of obtaining independent yet 
valid assessments of momentary attitude towards one’s own voice, 
directed that the items on the three forms be equivalent but not 
identical to each other. The instruments finally constructed proved 
to be most satisfactory and we present the forms here for the use of 
any colleagues who might wish to employ them in their own work. 

The forms, of course, were constructed to yield information about 
the subject’s own voice, and the concept to be rated by the polar 
adjectives is “my voice.” Nevertheless, there may be other concepts 
for which the items could be appropriate. Some students may wish 
to test whether these forms maintain their equivalence when they 
are used to describe other concepts. We are also suggesting our 
method of inventory construction as one that can assure equival- 
ence. Too often investigators cull items from the Osgood, Suci, and 
Tannenbaum book (1957) without pretesting their relevance or 
their factor structure for the concept they wish to study. We there- 
fore believe that whether or not the forms are appropriate for as- 


1 This study is part of a larger study on self-confrontation and is supported 
in part by Public Health Service Research Grant MH 07962, National Insti- 
tute of Mental Health. 
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sessing concepts other than “my voice,” the method of test con- 
struction recommends this presentation. 


is 7 Method 


From the Osgood, Suci, ‘and Tannenbaum book (1957) we first 
chose a pool of 70 items applicable on an a priori basis to the con- 
cept “my voice.” We arranged these 70 items in four orders: (1) a 
first random order, (2) reverse of the first random order, (3) a 
second random order, and (4) reverse of the second random order. 
Twenty-nine subjects were assigned randomly to each order mak- 
ing a total sample of 116 subjects. The subjects were sophomore, 
junior, and senior students at a midwestern university.2 They 
ranged in age from 19 to 24. Means, standard deviations, and the 
intercorrelation matrix for all 70 items over all 116 subjects were 
computed. 

We obtained a principal axis factor analysis of the intercorrela- 
tion matrix. The lower bounds of the multiple correlation coefficients 
were used as diagonal values. Four factors were extracted: the first 
contained 47 per cent of the common variance ; the second, 30 per 
cent; the third, 12 per cent; and the fourth, 11 per cent. These 
factors accounted for 17 per cent, 11 per cent, four per cent, and 
four per cent respectively of the total variance. There was too 
sharp a drop in the percentage of variance accounted for by suc- 
ceeding factors to warrant further extraction of factors. 

The four factor solution was rotated by the normal varimax ro- 
tation method, and after rotation, Factor I accounted for 33 per- 
cent of the common variance; Factor TI, 30 per cent; Factor III, 
24 per cent; and Factor IV, 14 per cent of the common variance. Of 
the total variance, Factor I accounted for 12 per cent; Factor II, 
11 per cent ; Factor III, nine per cent; and Factor IV, five per cent. 

We then chose groups of items that loaded almost exclusively on 
each of the four factors. We were able to choose 28 items for Factor 
I, 19 items for Factor II, 14 items for Factor III, and nine items for 
Factor IV. The items and their loadings are given in Table 1. In 
Table 1 defining loadings are considered to be loadings of .20 or 
above, but in our own actual procedure of selecting items we con- 
sidered .30 or above to be a minimum loading. 


2 We gratefully acknowledge the assistance of Mrs. Winifred Siegel who 
administered the forms and did all of the preliminary scoring. 


ret. 
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TABLE 1 
Normal Varimaz Factors 

Item val I Mi hs eae fo 
active-passive H 18 05 34 57 
sociable-unsociable —15 11-17 47 
positive-negative 62 17 29 07 50 
sharp-dull 61 09 —03 22 43 
bright-dark 61 —33 13 05 50 
constrained-free —61 —02 —05 15 40 
successful-unsuccessful 56 03 45 06 53 
fresh-stale 56 —03 34 18 46 
blatant-muted 56 20 —07 16 38 
optimistic—pessimistic 55 —07 09 -13 33 
graceful-awkward 54 —08 46 05 5l 
clear-hazy 53 —04 29 29 46 
wide-narrow 51 34 23 15 46 
refreshed-weary 49 —14 38 26 47 
colorful-colorless 48 14 48 20 52 
near-far 48 —19 14 03 28 
constricted-spacious —47 02 —05 20 27 
fast-slow 47 -15 —06 18 28 
interesting—boring 46 25 39 17 45 
believing-skeptical 45 —17 16 -13 27 
public-private 43 13 —06 21 24 
hot-cold 37 09 17 —05 18 
savory-tasteless 35 01 28 09 21 
long-short 35 11 10 14 16 
tangible-intangible 30 06 26 —06 16 
opaque-transparent —29 22 08 —17 17 
humble-proud —26 -24 -04 -12 14 
red-green 24 —05 -07 18 10 
rugged-delicate 05 82 03 —04 68 
masculine-feminine —04 79 —03 —03 63 
heavy-light 00 79 03 —01 63 
hard-soft —04 72 -22 07 58 
thick-thin —03 72 23 04 57 
deep-shallow 18 66 21 22 56 
strong-weak 39 60 30 13 6l 
youthful-mature —03 -59 —02 19 38 
large-small 41 58 —05 09 52 
sweet-bitter 13 —54 46 03 52 
light-dark 25 —52 06 05 34 
sacred-profane 02 —49 l4 —06 27 
rough-smooth —22 48 —43 -11 48 
severe-lenient —01 43 12 28 28 
wet-dry 20 30 27 25 26 
serious-humorous —09 28 14 -02 11i 
kind-cruel 16 —27 27 -02 17 
objective-subjective 09 26 01 18 ll 
sophisticated—naive 20 23 19 13 14 
nice-awful 25 —07 67 03 52 
pleasurable-painful 38 06 63 09 56 
beautiful-ugly 19 —03 58 19 41 
good-bad 34 13 5 —08 43 


eats 
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TABLE 1/ Continued 


Item No. Item } I Il It Iv 
16 calm-agitated f -14 06 63 —07 31 
15 rich-poor T 25 24 52 13 41 
18 sweet-sour 4 12 —39 49 Fe = 
10 important-unimportant 28 15 47 

4 d oe 05 —-36 44 14 34 
66 formed-formless 24 81 44 H sr 
12 reputable-disreputable 30 -21 88 

60 cautious-rash -09 -17 34 04 16 
47 angular-rounded 03 -04 -32 08 11i 
49 stable-changeable —=10 19 31 a 17 
53 unusual-usual —01 -04 -13 74 57 
39 complex-simple —06 07 13 69 50 
52 new-old 19 03 12 59 40 
61 ornate-plain 15 20 21 45 31 
61 straight-curved TUR 29 11 —40 26 
65 competitive-cooperative 23 31-15 89 33 
45 varied-repetitive 17 03 35 38 30 
59 Pungent-bland 26 21 11 35 25 
60 Sensitive-insensitive -17 —26 24 84 27 


Inspection of the items after the Totation solution suggests that 
Factor I be called an activity factor, Factor II a potency factor, 
Factor III an evaluative factor, and Factor IV a complexity factor. 
Factors I, II, and III are consistent with the factor analyses prev- 
iously reported by Osgood, et al. (1957), although the evaluative 
factor, which in most other studies accounts for the largest portion 
‘of the variance, was in our solution the third highest factor in 
terms of per cent variance, 


To construct the three equivalent sets of items for each factor, we 
originally planned to select indivj 
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form seven items for Factor Ië four items each for Factors II and 
III. Factor IV contained too few item to warrant inclusion. 

Table 2 presents the items selected for the three forms to rep- 
resent Factor I, the activity factor, their means, standard devia- 
tions, and factor loadings, as well as the mean, standard deviation, 
and factor loading for each form. The maximum difference in 
means among the three forms was .05; the maximum difference in 
standard deviations was .02; the maximum difference in factor 


TABLE 2 
Factor I—Activity 
——————————— eee 


Item Standard Factor 

Form No. Item Mean Deviation Loading 
A 2 sociable-unsociable 2.68 1.34 „64 
11 positive-negative 2.95 1.27 .63 
9 successful-unsuccessful 3.00 1.01 56 
14 fresh-stale 3.07 1.19 „56 
34 wide-narrow 3.53 1.28 52 
13 believing-skeptical 3.12 1.56 45 
68 public-private 3.94 1.45 43 
Average 3.18 1.31 55 
B 41 sharp-dull 3.25 1.12 62 
—25 free-constrained 2.80 1.58 „61 
42 blatant-muted 3.78 1.13 56 
20  clear-hazy 2.91 1.38 .53 
62 near-far 3.36 1.29 48 
—38  spacious-constricted 2.99 1.30 47 
63 tangible-intangible 3.37 1.43 30 
Average 3.21 1.33 52 
c 46 active-passive 2.98 1.40 .65 
17 bright-dark 3.26 1.28 61 
1 optimistic-pessimistic 3.10 1.49 55 
6 graceful-awkward 3.56 1.30 «54 
56 refreshed-weary 3.03 1.22 .49 
43 fast-slow 3.32 1.45 47 
58  interesting-boring 2.90 We Ye 46 
Average 3.16 1.33 54 


$ 
pee EN 

8 Out of a pool of 25 items with factor loadings greater than or equal to 
30, 21 were chosen for the three forms. Item #57, “colorful-colorless,” was 
omitted from Factor I because it had an equal loading on Factor III. Items 
#40, “hot-cold;” #55, “sayory-tasteless;” #35’ “Jong-short,” were also omitted 
from Factor I. With the exception of item #9, these three items had the low- 
est standard deviations of any of the items available for Factor I, indicating 
Jess discriminative power. Their factor loadings, too, are the lowest loadings 
among the pool of items with the exception of item #63. 
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loadings was .03. The average intercorrelation among the items for 
the first form was .32; for the second form was .25; and for the 
third form was .33. The average intercorrelation of the items for 
the first form with the items in the second form was .28; the items 
in the first form with the items in the third form was .33; and the 
items in the second form with the items in the third form was .26. 

Table 3 presents the items for each of the three forms for Factor 
II, the potency factor, together with means, standard deviations, 
and factor loadings for both items and forms. For the three forms 
on Factor II, the maximum difference among means is .07; among 
standard deviations is .06; and among factor loadings is .03. The 
average intercorrelation among items is .41 for the first form; .32 
for the second form; and .32 for the third form. The average in- 
tercorrelation between items of the first form and the second form is 
42; between the first form and the third form is .38; and between 
the second form and the third form is .34. Of the 14 available items 


for Factor II having loadings of 30 or above, 12 were used in the 
three forms.* 


TABLE 3 


Factor II—Potency™ 
SSS. 0 


Item Standard Factor 
Form No. Item Mean Deviation Loading 
$$ AT Deviation Loading 
A 37 rugged-delicate 3.46 1.32 
$ i .82 
32 hard-soft 3.88 1.54 .72 
23 strong-weak 3.00 1.49 -60 
—5 dark-light 3.12 1.32 -30 
Average 3.36 1.42 -64 
B 31 heavy-light 3 
30 deep-shallow es vie 1 
-54 mature-youthful 2.91 1.68 «59 
24 — severe—lenient 3.98 1.33 43 
Average 3.42 1.48 63 
(0) S masculine-feminine pg 2.16 79 
—44 bitter-sweet 3.37 Lol á 
—21 profane-sacred 3.33 1.16 a 


Average 


| 


4Items #29, “rough-smooth,” and #33, 
we were least able to match them on 
other items, 


“thick-thin,” were discarded be 
cause 
means and standard deviations with 
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Table 4 presents the items selected to represent Factor III, the 
evaluative factor, means, standard deviations, and factor loadings 
for both items and forms. The maximum difference in form means 
was .04; maximum difference in form standard deviations was .08; 
and the maximum difference in form factor loadings was .04. We 
omitted two items from Factor III: items #47, “angular-rounded,” 
and #49, “stable-changeable.” These items had the two highest 
means and the two lowest loadings among the pool of possible 
items, making them difficult to match. The average intercorrelation 
among these items for each of the three forms are respectively .26, 
.21, and .25. The average correlation between the items of the first 
form and the items of the second form is .29; between the first 
form and the items in the third form .26; and between the items in 
the second and third forms .27. 


TABLE 4 
Factor II1I—Evaluative 
Item Standard Factor 
Form No. Item Mean Deviation Loading 
A 7 pleasurable-painful 2.73 1.06 -63 
8  beautiful-ugly 3.60 0.96 .58 
4 clean-dirty 2.72 1.25 44 
50 cautious-rash 3.35 1.41 -34 
Average 3.10 1.18 .51 
B 22 good-bad 2.91 1.34 .54 
18 sweet-sour 3.46 1.05 .49 
10 important-unimportant 2.99 1.20 47 
66 formed-formless 2.95 1.26 44 
Average 3.08 1.22 49 
Cc 19 nice-awful 2.96 1.22 67 
16 calm-agitated 3.38 1.47 53 
15 rich-poor 3.27 1.12 .52 
12 reputable-disreputable 2.87 1.19 .38 
Average 3.12 1.26 53 


Although the selection of items for the three equivalent forms 
was done on a form matching, rather than individual item match- 
ing, basis it can be seen that for all three factors the form means, 
form standard deviations, form factor loadings, and form average 
intercorrelations are strikingly close. 

We tested the equivalance of the three semantic differential in- 
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ventory forms in an experiment employing 20 Ss in an experimental 
group and 20 Ss in a control group. Ss in the experimental group 
tape recorded a standard passage and then filled out Form A of the 
semantic differential (condition I). Then they listened to a five- 
second sample of their recorded voices and immediately completed 
Form B of the semantic differential (condition II), Five minutes 
later, after an interpolated audiometric test, Ss completed Form 
C of the semantic differential (condition III). Ss in the control 
group followed the identical procedure except that they completed 
Form A after each of the three conditions. 

Figure 1 presents the results. Ss in the experimental group— 
those who completed the three equivalent semantic differential 
forms—show an experimental effect of listening to their own voices: 
there is a significant shift in the evaluative and activity scales from 
conditions I to II, and in the potency scale from conditions II to 
TII. These shifts are attributable to the experience of listening to 
their own voices. For Ss who completed the same form for each of 
the three conditions no such experimental effect appears. The psy- 
chological implications of these results are discussed in another 
paper (Holzman and Rousey, in press). Here we wish only to indi- 
cate that repeated responses to the same semantic differential form 
reflect primarily the repeat reliability and error variance of that 
form, making the form relatively insensitive to momentary atti- 
tudinal changes. The measurement of such fleeting shifts in atti- 
tude by means of a semantic differential inventory would therefore 
Tequire construction of equivalent forms, in the manner suggested 
here, to control the serial effect of repeatedly using the same form. 


Summary 


: We constructed three equivalent forms of a semantic differential 
inventory relevant to the concept, “my voice.” The effectiveness of 
the use of equivalent forms for Measuring quick attitudinal shifts 


i Caption for Figure on opposite page. 
oh aa we omes differential judgments of “My Voice” on the evalu- 
mediately after and 8 minutes ater teas) oS baer ie all 

i -i g eir i 
of Ss: those using 3 different but equivalent forms kit ue ig ine 
form for each condition. 
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was demonstrated. Whether the forms maintain their equivalence 
when used to describe the meanings of other concepts is a matter 

_ for empirical test. We recommend this method of semantic differ- 
ential inventory construction as one that is more precise than 
simply culling items from a large pool on an a priori basis, 
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THE EFFECT OF SITUATIONAL VARIABLES ON 
THE MEASUREMENT 
OF ACHIEVEMENT MOTIVATION 


RICHARD E. CARNEY 
California Western University? 


Tum present paper will consider some of the traditional dis- 
tinctions between objective and projective measures of achieve- 
ment motivation, will present evidence which casts doubt on such 
distinctions, and will offer a tentative basis for integrating those 
two approaches into a common framework. The discussion will be 
limited for purposes of economy to a single pair of measures which 
purport to measure a single motive; however, it is anticipated that 
some generalization will prove possible. 

Our example of an objective measure of achievement motivation 
is the Achievement-Orientation (AO) scale from the California 
Psychological Inventory (CPI) (Gough, 1957). Our example of a 
projective measure is the need Achievement measure (n Ach) from 
the Thematic Apperception Test (TAT) (McCelland, Atkinson, 
Clark, and Lowell, 1953). Each of these measures has substantial 
evidence for validity (Atkinson, 1958; Carney, 1961, 1963a; 1964; 
Carney, and McKeachie, 1963), yet the association between them 
has been elusive as has so often been the case when objective and 
projective measures of the same variable are compared (Fiske, 
1963). Such lack of relationship and other considerations have 
lead McClelland (1958) to conclude that the self-descriptive and 
the fantasy measures do not measure the same thing. 

A brief bit of preliminary theorizing may help to resolve some 
confusion in the area of projective versus objective measurement. 


1 Formerly at Indiana University (Northwest Campus) and Drake Univer- 
sity. 
675 
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After which consideration will be given to empirical correlations 
which have been computed between n Ach and AO under a wide 
variety of conditions, and the latter portion of this paper will be 
devoted to a discussion of the variables which affect the n Ach-AO 
correlation (r), 


Concepts of Personality 


Personality exists only as a process of interaction between the 
organism and its environment (Fiske, 1963). Every behavior, how- 
ever it may be observed, has originated in this same process. It is 
time we stopped even talking as if there were some independently 
functioning “part” of personality to be projected into behavior. 
Any concept of a motive capable of indeterminate freedom of ac- 
tion is the true heir of vitalism, and we should be, by now, finished 
with such concepts, Unfortunately, this is not the case (Carney, 
1963b, Immergluck, 1964). It has become almost traditional to as- 
sert that the projective situation permits “greater freedom of re- 
sponse” (no real indeterminancy intended) (e.g. Lindzey, 1961, pp. 
42-46). In practice such statements refer to the fact that a nebu- 
lous pattern of stimuli leads to a highly variable set of responses. 

A strictly deterministic model requires that all behavior upon 
which inferences of motivation are based be a joint interactive 
function of the organism and its environment. If we accept Atkin- 
son’s (1958) formulation of Motivation = f (Motive X Expectancy 
X Incentive) and his definition of a motive as a relatively stable 


disposition to act, then the behavior we observe (aroused motiva- 
tion) is by definition Jointly determined by the action of a particu- 
lar set of stimuli with a 


: ; particular organismic structure. Those 
working with the n Ach measure have been acutely aware of situ- 
ational influences which ¢ 


A an and do change the responses to the 
TAT (Atkinson, 1958). Lindzey (1961) provides a concise sum- 
mary of the complexities Which enter into the measurement context, 
n the following discussion will explore a few of these complexi- 
jes. 


Techniques of M. easurement 
7 Questionnaires, The questionnaire technique, 
inates much of the personal interaction from 
By using a large number of short, well stru 


if well done, elim- 
the testing situation. 
ctured, printed items 
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which refer to presumably familiar and significant life situations, 
questionnaires tend to control the effective stimulus matrix and to 
also provide a simple unambiguous response. Hopefully, the S’s 
attention is involved primarily with the presented items and is 
thereby largely removed from other situational influences. All that 
is done is to insure a reasonable knowledge of what it was that 
participated on the environmental side of the O X E interaction. 
This control can in no way influence the predispositions or motives. 
It is interesting that the questionnaire is attacked because the re- 
sponses of the Ss are “too subjective” (Watson, 1959). Sometimes 
this critique comes from those who simultaneously argue that fan- 
tasy is the proper datum (McClelland, 1958). When faced with a 
questionnaire item there is little that an S can do (if he participates 
at all) except engage in cognitive and imaginative processes (fan- 
tasy?). The need to construct “lie” scales and the discovery of the 
influence of the Social Desirability factor (Edwards, 1959) testify 
that a seemingly barren X mark can be preceded by remarkable 
thought processes fully open to unconscious influences. Thus there 
is nothing intrinsically different about the O side of the O X E 
interaction, only the form of the final response varies. Some in- 
vestigators like Xs and some prefer words, but this preference 
probably indicates more about the investigators than the methods 
they employ (Fiske, 1963). There is no logical reason to exclude 
responses to questionnaire items from the valid measures of mo- 
tivation. 

TAT. Behavior elicited by the TAT pictures is no less a function 
of the O X E interaction. The major formal difference between the 
questionnaire and the TAT is that the test stimuli of the TAT pro- 
vide fewer effective stimuli. By intention, the test pictures are ren- 
dered ambiguous, with only the minimum structure needed to furn- 
ish a response setting. Since, by the present analysis, there must be 
stimuli in the measurement situation which provide the activating 
environment, the net effect is to force the S to find cues in aspects 
of the testing context other than those directly provided by the 
TAT. It is not immediately obvious why such stimuli should be 
expected to lead to “fantasy” responses less biased by social factors 
than responses made in self-description (McClelland, 1958). The 
crux of the matter here is: Does one “escape” from the determining 
predispositions learned from society when composing a story about 
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others, or does one use essentially the same processes as when com- 
posing a story about oneself? The object of the tale has superficially 
changed, but if it really had changed there would be little use in 
analyzing fantasy to infer the motivation of the storyteller. Our 
stories about ourselves are usually fantastic enough! 

Behavior resulting from projective testing is then, derived in 
the same manner in principle as that from objective testing. There 
is no inherent reason why it cannot also be a valid measure of 
motivational differences—provided that some systematic knowl- 
edge is available about the stimuli which actually elicited the be- 
havior, (See Holtzman, Thorpe, Swartz and Herron, 1961, for an 


approach which seems to provide some such knowledge about ink- 
blots.) 


A Synthesis 


What the fabled difference between objective and projective 
Measurement resolves itself into, then, is that objective techniques 
produce a relatively standardized stimulus situation which elicit 
relatively standardized behavior, while projective techniques often 
Produce non standardized behavior which is elicited by non stand- 
ardized extra-test situational factors, There is no “freedom of 
Tesponse” which produces “interesting and significant deviant re- 
Sponses” in the projective situation. There is simply less control 
and considerable ignorance as to which stimuli were effective. Table 
1 summarizes the above discussion, 


e TABLE 1 
he Influence of Test Stimuli, Context and Mi otivation on Ri 
to Objective and Projective Tests ties 
ts 
0 ae 
Variation Objective Projective 
Test stimuli high low 


Objective-projective as used here is, of Course, a dichotomy ap- 
plied to a continuum, and the n Ach measure was selected as the 
projective example expressly because its users have gone farther 
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than most in sorting out the confounding effects of situational 
variables. The moral for measurement seems none-the-less clear, 
unless we are specifically interested in the reaction of an S to the 
total test context (as we well may be in clinical practice), we are 
better off to control that context. If we are interested in exploring 
reactions to the context, there is no easy way out of the necessity of 
a rigorous and detailed definition of every significant facet of that 
context (see Lindzey, 1961, pp. 323-328; Fiske, 1963). Such an ex- 
ploration is ordinarily expensive and quite time consuming. Self- 
description is so much easier, and in principle the same. Why not 
use it when we can? 


Some Implications 


TAT responses are assumed to be largely determined by the im- 
mediate situation. It is when the situation extra to the test provides 
cues for achievement motivation that the TAT and questionnaire 
measures should coincide. Ss obtained from the classroom should 
have a general set which reflects the subject matter content, in- 
structor personality, and classroom procedures and this set should 
systematically contribute to the obtained TAT responses. Ques- 
tionnaire responses can also be manipulated by varying the set of 
the S (Carney, 1964), but are assumed to be relatively less af- 
fected under ordinary circumstances than the TAT responses. 

McKeachie (1961) has shown that Ss high in achievement mo- 
tivation prefer classroom situations where they can assume some 
responsibility for their own behavior. Carney (1961) presents evi- 
dence of interactions between personality and subject matter on 
the attitude of the S toward a course. 

The remainder of this paper documents some of the factors 
which influence the correlation between CPI and TAT measures 
of achievement motivation. The following outcomes were expected: 

1. A real but highly variable r exists between AO and n Ach. 

.2. The AO-n Ach r will be affected by salient features of the to- 
tal situation in which the measures are obtained—such as; the 
motivation of the instructor, and the “atmosphere” of the class- 
room. 

3. The n Ach measure will be most sensitive to the situational 
variables, and it is this sensitivity which most probably accounts 
for the lability in the relationship between n Ach and AO. 
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Procedure 


Sample. The Ss were drawn from a Mid-Western college sample 
which is described in detail elsewhere (McKeachie, 1961; Carney 
and McKeachie, 1963). Over 800 Ss in 70 separate classes (inelud- 
ing Psychology, French, and Mathematics courses) at two uni- 
versities were involved. 

Measures. n Ach and AO measures were derived by standard 
techniques (Veroff, et al.; 1960, Gough; 1957, Carney, 1961). 
Measures of n Ach, and need affiliation (n Aff) were obtained from 
the instructors by using the French Test of Insight, and Bales scor- 
ing for group interaction was done for some classes (McKeachie, 
1961). 

Statistical Procedures. All relationships were plotted on scatter 
diagrams and visually inspected for factors which might influence 
the value of a Pearson product-moment correlation (r), such as 
non-linearity, extreme skewness, mean differences, and direction- 
ality of the relationships in sub-groups. 

According to the above inspection, 7s were computed and com- 
pared. The 2’ transformation was used when averaging or testing 
for significant differences between rg (Edwards, 1960). The trans- 
formed unweighted rs were used for all comparisons between r and 
other variables. 

The unweighted means of the various sub-groups were used for 
all comparisons between level of motivation and other variables. 


Analysis of variance and other tests of significance were carried 
out according to Edwards (1960). 


Results 
Correlations between n Ach and AO 


obtained by averaging the rs 
though there is a low, signific: 
females haye an insignificant 


hc 
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mean absolute n Ach-AO r indicate that there is more to this situ- 
ation than is immediately apparent. Tables 3, 4, 5, and 6 illustrate 
this point in detail. 


TABLE 2 


Correlations between n Ach and AO for Total Sample 
(University of Michigan, 1957; Drake University, 1962) 


Mean 
Sex r df absolute r df 
Male .109* 400 .305** 203 
Female —.040 456 .351** 362 
Total 075 856 -330** 645 


*p <.05. 
** p < .01, two tailed tests. 


TABLE 3 
Correlations between n Ach and AO for French Classes 
(University of Michigan, 1957) 


Mean 
Sex r n absoluter n Classes df Test between rs 
Males 
Male Inst. .233 31 — pooled MI-FI is n.s. 
Female Inst. —.073 2 — pooled 
Females 
Male Inst. .036 93 .427** 9 66 x? = 19.86 p < .02 
(between classes) 
Female Inst, —.269* 50 +242 vf 29 MI-FI is n.s. 
Totals 
Male Inst. .099 124 .373** 9 94 z between 
Female Inst. —.185 71 178 7 47 absolute mean 
Male Ss .115 52 171 pooled 46 rs for both 
Female Ss  —.054 143  .375** 16 95 MI-FI and MSs 
All Ss —.020 195 310** 16 141 FSs = 2.1 p < .05 


Note.—In several classes there were insufficient ns to permit the computation of meaningful 
rs. This was true for the males in all French classes. The loss of Ss from small classes will make 


both the number of classes and of subjects change from table to table. 


*p < .05. 
** p < .01, two tailed tests for the mean absolute rs. 


TABLE 4 
Correlations between n Ach and AO for Mathematics Classes 
(University of Michigan, 1957) 


Absolute Test 
Sex r n mean r df n Classes between rs 
- Males .254* 85 .261* 61 8 Classes, n.s. 
Females 291 48 821 27 7 Classes, n.s. 
Total .264** 133 .279** 88 8 M-F, n.s. 


tp <05 _ 
** p < .01, two_tailed tests for the mean absolute r. 
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TABLE 5 


Correlations between n Ach and AO for Psychology Classes 
(University of Michigan, 1957) 


Absolute 
Sex r n meanr df n Classes Test between rs 
Males —,163 75 .577** 56 6 Classes, x? = 40.61, p < .01 
Females .004 187 .325** 119 6 Classes, x? = 16.70, p < .01 
Total —.050 212 .413** 175 6 M-F, n.s. 


** p < .01, two tailed tests for the absolute mean r. 


TABLE 6 


Correlations between n Ach and AO for Psychology Classes 
(Drake University, 1962) 


O OOO 


Absolute 
Sex r n meanr df n Classes Test between rs 


Males  .192* 138 .249** 120 6 Classes, x? = 17.33, p < .01 
Females —.217* 130 .363** 121 3 Classes, x? = 11.74, p <.01 
Total —.014 268 .286* 241 6  MF,z=33l,p. <.01 


Note.—Only 3 instructors participated. One instructor taught 4 of the 6 classes, Only 3 classes 


had a sufficient n of Female Ss to permit computation of r; each of these classes was taught by a 
separate instructor, 


*p < 05. 
*p < .01, two tailed tests for the absolute mean rs. 


The Effect of Type of Course and Sex 


French Classes. Table 3 presents the results of the French classes 
at the University of Michigan. Since there were both male and fe- 
male instructors for these classes, the data have been presented ac- 
cordingly. The n of male Ss was too small to permit computation 
of r for each class, so all of the male Ss were pooled for male in- 
structors (MI) and female instructors (FI) respectively. The fe- 
male Ss respond to the female instructors fairly uniformly yielding 
a homogeneous set of —rs between n Ach and AO. The male in- 
structors produce both greater variability (and a zero order r) and 
a higher absolute mean value of r, The male Ss show a pattern of 
signs for the n Ach-AO rs similar to that of the females, but seem 
to have a higher -+r and a lower —r for the MI and FI categories 
respectively. Again, the r for all Ss and all classes is near zero. This 
fact conceals both the effects of the sex of the instructors and the 
effect of being in an individual class. The substantial value of the 
absolute mean value of ris also concealed. 

Mathematics Classes. Table 4 shows the results for the Math- 
ematics classes at the University of Michigan. In this case the n 
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Ach-AO rs for both sexes and over all classes were consistent and 
positive. Both the total r and the total absolute mean r are sig- 
nificant and in good agreement with each other. 

Psychology Classes. Tables 5 and 6 show the results for the 
Psychology classes at the University of Michigan and Drake Uni- 
versity respectively. There is striking disagreement between the 
rs within each table and between tables. As was the case with the 
MI French classes, total rs indicate zero order relationships while 
mean absolute rs show substantial relationships. The signs of the 
rs reverse for the males and females at the different schools, and 
the rs for both males and females are significant at Drake but 
neither reach significance at Michigan. This latter result is prob- 
ably due to the fact that one instructor taught four classes at 
Drake introducing an element of consistency which was lacking at 
the University of Michigan where six instructors each had one 
class (one lecture, and two discussion groups) .? 

To summarize the results of this section, the size and direction of 
the r between n Ach AO depends on the sex of the S, the sex of the 
instructor, the course content, and the particular class in which the 
S is enrolled. However, the mean absolute r is quite stable across 
the above variables and indicates a population value for r between 
n Ach and AO of .330. 


Effect of Instructor Motivation on the n Ach-AO r 


Effect of Single Motives. Table 7 shows the source of some of 
the variability in the n Ach-AO r within courses—the motivation of 
the instructor from whose class the Ss were taken. For this com- 
parison the n Ach-AO rs were ordered with the largest —r at the 
low end of the scale and the largest -+r at the high end of the 
scale. This ordering was then correlated with the motive scores of 
the instructors. Both male and female Ss tend to have —rs in 
classes where the instructors are highly motivated for either n Ach 
or n Aff. When the instructors are low in motivation, the n Ach- 


2R. J. Roper in an unpublished study compared AO and n Ach in three Intro- 
ductory Psychology classes at California Western University during the winter 
quarter of 1965. These classes were taught by separate instructors and con- 
tained 33 male and 44 female Ss. The absolute mean r over classes was .259 
(p < .05, 68 df). Although the sub-group rs were statistically homogeneous 
(total r = —.195, p < .05, 75 df), one class had r = .121 (27 df) and the aver- 
age r for the other classes was —.340 (p < .05, 42 df), These data extend the 
basic findings to still another school and section of the country. 
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AO r tends to be positive, Intermediate levels of instructor moti- 
vation produces zero order rs. There is an exception to this rule in 
the French classes where the female Ss seemed to respond differ- 
entially to the sex of the instructor. 

TABLE 7 


Correlations between Instructor Motives and N Ach AO Correlation 
(University of Michigan, 1957) 


n values n values 
Sex Courses n Ach ofr n Af ofr 
Males All —.481* 16 —.764** 16 
Females Psych & Math — .530* 13 — .347* 27 
French +248 14 
Total —.503* 29 —,529%* 43 


$p <.01 
TABLE 8 
The Correlation between AO and n Ach as a Function of Bales Scores 
of Classroom Interaction 


(University of Michigan, 1957) 
To i a i 
Class Sex Bales Score n Classes 
Freh Gee Sma M 


Teacher- Student- Warmth 
(females only) Led (TL) Led (SL) (W) 
Pesan, — .750* 054 +342 9 
? ‘emale —.518 384 009 6 
Mathematics a 304 —.464 — .759* 7 
543 —.614 — .829* 6 
Psychology M — .786* -557 -989%* 6 
F —.443 500 543 6 
oe order correlations, 
"p <o 
} TABLE 9 
Correlations between Motives of Instructors and Mean Class Levels 
of Motive Scoring 


(University of Michigan, 1957) 


n Ach Classes nA Classes 
— n Ach Inst. n Af Inst. n Ach Inst, n vr ast. Clases 
leg +202 -100 —.444* 
penal One reas ‘Aga me 26 
TER oe +183 065 —1253* 28 
ote.—" mal female 
different G = 827, p< ona tot bet yacht instructor and casos are significantly 
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Effect of Paired Motives. An analysis of variance was carried 
out using the transformed rs from each sub-group of males and fe- 
males (one per sex from each class) as the dependent variables 
and the motivation of the instructor as the independent variable. 
Classes were grouped according to the standing of the instructor 
above or below the medians of n Ach and n Aff. The motive scores 
were taken two at a time and High-High, High-Low, Low-High 
and Low-Low classifications were formed. This analysis should be 
considered only suggestive since a small number of sub-groups 
entered each classification, and details of the analysis will not be 
presented. However, two findings were consistent in every compari- 
son and should be investigated further. The pattern of correlations 
over motive classifications was different for each type of course con- 
tent producing a significant Course X Motive groups interaction for 
each combination of motives. Also the average rs were near zero 
for all motive classifications except the Low-Low one where in each 
combination of motives the average r was positive and at a level 
closely approximating the estimate for the population r given above. 


Effect of Classroom Procedures on the AO-n Ach r 


Bales Scoring and the AO-n Ach r. Nine ratios were obtained 
from the interaction patterns in each of the University of Michigan 
classes; Teachers Task (TT), Teacher Assertion (TA), Teacher/ 
Student Ratio (T/S), Student Task (ST), Student Volunteered 
Acts (SV), Teacher Positive Socio-Emotional Acts (T+), Student 
Positive Socio-Emotional Acts (S+), and Student Tension (TE) 
(McKeachie, 1961). The average scores for the French, Mathe- 
matics, and Psychology courses showed quite consistent patterns 
over the nine ratios. On the basis of these patterns and logical con- 
siderations, the TT, TA, and T/S ratios were summed to obtain 

~ a Teacher Led Class (TL) score; the ST, SA, and SV ratios were 
combined to obtain a Student Led Class (SL) score; and the T-++, 
S-L, and inverted TE ratios were summed to obtain a Warmth 
(W) score. 

Mathematics classes tended to be strongly teacher-led, had the 
lowest SL scores and were lowest in W. The French classes were 
strongly student-led, lowest in TL and highest in W. Psychology 
classes were moderate in every score, but most closely resembled 
the French classes. 
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Table 8 shows the AO-n Ach rs as a function of the Bales scores, 
The pattern for the males and females is similar in each case. In 
both the French and Psychology classes the classes strongly led by 
the teacher tended to produce —rs, while those Psychology classes 
high in SL and W tended to have +rs. The Mathematics classes 
presented a quite different pattern with the classes higher in TL 
having -+rs and the classes high in SL and W having —rs, 

The overall pattern Seems to be that classes which fit the general 
pattern for their type of course produce +-AO-n Ach rs, and classes 
which do not fit the overall pattern produce —rs. The small num- 
bers of classes involved make this conclusion most tentative. 


Effect of Situational Variables on Level of Motivation. 


ent (z = 3.53, p < .001). 
Over all classes a mean r 


gae mean n Ach were correlated (males, r = 631, p < .01, 16 df; 
emales, r — — 233, 29 df; male versus female r, z = 3.58, p < 


—— 
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a role in determining test behavior. The evidence here is that this 
was the case for n Ach and not for AO. Such a finding is consistent 
with the present analysis, but certainly not conclusive. A carefully 
designed series of parametric experiments are needed to determine 
the conditions under which the AO and the n Ach measures are af- 
fected by variables outside of the immediate testing stimuli. 

A final question remains as to the interpretation of the changes 
in the r between AO and n Ach. If it is assumed that the meaning of 
the AO measure is relatively constant, but that its application to 
a given situation depends on the response of the S in that situation, 
then some measure of the situational response is needed. n Ach 
seems to be a good possibility. At the high extreme of motivation, 
instructors would tend to impose their own motives on the class and 
by doing so would lessen the chance of class members to satisfy 
their achievement motives (McKeachie, 1961). At the low extreme 
of motivation the instructor would tend to leave a vacuum into 
which the Ss could rush if they were so motivated. As would be 
predicted if n Ach reflects the degree to which the S is responding 
to his immediate chances of structuring the environment, n Ach 
and AO are positively correlated in classes which have low levels 
of instructor motivation, The n Ach-AO rs tend to zero in classes 
with intermediate levels of instructor motivation. The level of 
scoring on n Ach is lowered (for males) by high levels of instructor 
motivation. 

The Bales scores tend to confirm the above analysis in the Psy- 
chology and the French classes. However, the Mathematics classes 
produced the most consistent +40 — n Ach rs, and these were the 
most strongly teacher led classes in the study. The sex of the in- 
structor also seemed to play a role in the French classes. What the 
S expects the interaction pattern in a particular course to be may 
be important. This is another testable hypothesis in search of a 


test. 
Litwin and Ciarlo (personal communication, 1963) have found 


| that n Ach predicts situations in which the S has maximum op- 


portunity to structure the task, while a questionnaire measure of 
achievement motivation predicts tasks which are highly structured. 
A profitable line of future investigation should be to combine ma- 
nipulation of the S’s test taking set toward the TAT (fostering the 
belief that S can structure or not structure the context within 
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which the test is given), with variation in the structurability of a 
performance criterion, 

The variable of. sex of the § Temains a strikingly effective but 
puzzling one. There is a tendency for females to have mirror image 
patterns to that for males, but this is not always the case. Females 
in our“culture may define achievement motivation as “conforming — 
to the requirements of the defined task.” (Carney, 1961, 1963a, 
1964). If the situation is high in cues for achievement (and well 
structured) females may respond most highly in terms of achieve- 
ment motivation. Evidence for this proposition in the present data 


is high in n Ach, and a frequent tendency to have relationships 
‘opposite to those for the males. However, in the case of the Bales 


Scores the obtained 7s were similar for both sexes, The mysterious 
female is still with us, 


Projective and 
tive under the Proper conditions. What these conditions are needs 


Measures can be excellent indicators of 
Situational variables, 
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Boru the self report and the self concept are currently among 
the most popular subjects for psychological research. Wylie (1961) 
for example, has reviewed 463 such articles. Many of the studies 
reported by Wylie as well as many more reported more recently 
have utilized the self report as a direct measure of the self concept. 
This use of the two terms as though they were synonymous has 
been vigorously challenged by Combs and his associates (Combs 
and Snygg 1959; Combs and Soper 1957; Combs, Soper, and Cour- 
son, 1963). They point out that the self concept is an internal or- 
ganization of the individual’s perceptions about himself, whereas, 
the self report is a behavior representing what the individual is 
willing and able to say about himself when he is asked to declare 
his position. They insist that the self concept must be approached 
through some form of inference based on the individual’s behavior. 
This research was designed to explore the relationship of self re- 
port to inferred self concept in the hope of shedding some light upon 
this confusion. 


Problem 


Theoretically, several variables may operate to produce an un- 
determined amount of distortion in the self report when it is used 
to measure the self concept. Combs and Snygg (1959) and Combs 


1 Based on a dissertation submitted to the University of Florida. The author 
expresses his gratitude to Arthur W. Combs for his generous assistance in 
directing the study. 
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and Soper (1957) list factors which may influence an individual’s 
self report: 

1. The clarity of the subject’s awareness, 

2. The availability of adequate symbols of expression. 

3. The willingness of the subject to cooperate. 

4. The individual's feeling of personal adequacy. 

5. The individual’s feeling of freedom from threat. 
6. The social expectancy. 

If these factors do, in fact, interfere with the reliability of the 
self report, self concept study with such instruments. will produce 
questionable conclusions, 

According to theory, the self concept is the individual's personal 
organization of ‘perceptions about self. Since this organization af- 
fects the individual’s behavior the self concept can be studied by 
reversing this process and inferring the nature of the self from 
behavior. The method used to elicit behavior should minimize the 
possibility of influence by factors which are claimed to affect the 
self report. 

The general aim of this study was to examine differences between 
the self reports made by children and their self concepts as in- 
ferred from their behavior. These hypotheses were formulated: 

1. The self report and the inferred self concept will show low 
correlation when each is obtained in a way that protects the 
anonymity of the subjects. 

Unsigned self reports will not correlate highly with self re- 

ports which the subjects are required to sign. 

. There will be high correlation between inferred self concepts 

when subjects believe that their responses are held in confi- 

dence and inferred self concepts obtained when subjects know 

that their responses will not be held in confidence. 

The correlation between signed self reports and inferred self 

Concepts obtained when subjects know that their responses 

are not confidential will be lower than the correlation between 

unsigned self reports and inferred self concepts made when the 

subjects believe that their responses are confidential. 

She pees > well acquainted with the subjects, will 
y recognize individuals from inferences 


made by the experimenter th x 
Teports. an from the subjects own self 


5 


wo 


fo 
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Method 


Subjects 


Subjects cooperating in this study were thirty sixth grade chil- 
dren enrolled in a public school in an urban area. 


Research Instruments 


The study was limited to five areas of the self concept which 
could be expected to be expressed in behavior in the school environ- 
ment: the self, the self in relation to others, the self as achieving, 
the self in school, and the physical self. Two research instruments 
were developed as means of testing the hypothesis. 


Self Report-Inferred Self Concept Scale 


A rating scale was devised which would serve as a self report 
instrument filled out by the children as well as a score sheet for 
recording the inferred self concept from the picture story test. It 
was a five-point scale comprised of six items for each of the five 
areas of the self concept which were being investigated. The items 
were arranged in pairs of positive and negative statements at 
either end of a continuum. Some of the items were reversed so that 
positive and negative statements did not always run in the same 
direction. The rating scale, then, had thirty items. 


Items 

1. I’m good in school work I’m not good in school work 
2. Mostly I have good ideas My ideas are poor 

3. I’m a worthwhile person Tm not a worthwhile person 
4. I’m pretty strong I’m not too strong 

5. Most people trust me Most people don’t trust me 
6. Teachers like me pretty well Teachers don’t like me too much 
7. I can do most things well I do very few things well 

8. I’m a happy person I’m an unhappy person 

9. I’m healthy I’m not too healthy 
10. I’m popular Tm not too popular 
11. I’m a good reader I’m not a good reader 


m 
i<) 


. I'm a hard worker 

. I’m very shy 

. I don’t get tired quickly 

. Other people find me interesting 
. I work well with others in school 
. Pm pretty brave 


ee 


Tm not a hard worker 

Tm not shy 

I get tired quickly 

Tm not too interesting to others 

I don’t work well with others in school 
Tm not too brave 
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18. I’m pretty smart 
19. I'm not tall enough 
20. Most people are fair with me 
21. I don’t do so well in class 
ussions 
22.1 nhs most of my problems 
wel 


23. I’m a helpful person 

24. I'm good looking 

25, Most people are hard for me to 
get along with 

26. I'm mostly happy in school 

27. I can usually finish what I start 

28. I'm proud of me 


29. I handle my body well in sports 
and games 


I’m not very smart 

Tm tall enough 

Most people are unfair with me 
I do well in class discussions 


I can’t handle my problems very well 


Tm not too helpful 

T'm not too good looking 

Most people are easy for me to get 
along with 

I’m mostly unhappy in school 

I never finish most things 

T'm not too proud of me 

I don’t handle my body well in sports 
and games 


30. I’m not often sorry for others 
Picture Story Test 


Tm often sorry for others 


ed as a technique for eliciting be- 
ould base inferences about the self 


Procedure 
Phase I 
The project was introduced to the class of thirty sixth graders 
as an effort to learn how boys and girls regard themselves in school 


helping children learn better, 
For the initial admi: 
Were promised anonymity, 


80 that identification was, in fact, 


Possible. 
The picture story test was Presented to each child in a manner 
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which implied confidentiality. The children were told not to tell 
their names to the examiner and they were assured that the stories 
they supplied would not be revealed to anyone. Actually a system 
was worked out with the teacher whereby identification could be 
made. 

After both the self report and the picture story test had been 
administered, the author told the class that the experiment was 
finished and it had furnished valuable information about children. 


Phase II 


The object of the second phase of the experiment was to observe 
the influence of social expectancy on both the self report and the 
inferred self concept. 

The teacher informed the class that she knew the author had 
learned much from his experience with them. She recognized that 
she could not have access to the information they had given because 
it was confidential. However, she thought that similar data about 
individuals would be helpful to her as a teacher. The children 
agreed to take the tests again. 

Before administering the self report and the picture story test a 
second time, much stress was placed on the idea that information 
gained would be given to the teacher. It was, therefore, of great im- 
portance that the children sign their self reports and that they ex- 
ercise great care in completing them. Also each person was in- 
formed when he appeared for the picture story test that his story 
material would be interpreted to the teacher. Thus, it was assumed, 
the subjects were made highly aware of the social expectancy 
factor. 


Phase IIT 


In the third phase of the experiment it was reasoned that, if the 
inferred self concept was a more accurate reflection of the indi- 
vidual, a teacher who knew the subjects well would see ratings 
based on inference as more descriptive of the person than his own 
self report. Data were used from the second phase of the experi- 
ment when social expectancy was stressed. This choice was based 
on the idea that in the actual school situation information given by 
pupils on personality tests would probably be influenced to some 
degree by social expectancy. Therefore, the ratings from phase two 
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more closely approximated what would be yielded in an actual 
attempts to learn about children in the classroom. 

The results of the signed self reports were reproduced on fresh 
rating scales by use of a rubber stamp. The results of the second 
administration of the picture story test were reproduced in the 
same way. This made it impossible for the teacher to know whether 
she was examining a subject’s self report or his inferred self con- 
cept. The pile of rating scales containing paired reproductions of 
the self report and the inferred self concept was given to the teacher. 
She was asked to select the one rating scale for each subject which 
she believed to be the better description of him. 


Results 


To determine the relationship of unsigned self reports to the 
self concept inferred from picture stories obtained under anony- 
mous conditions, Pearson correlations were computed for each of 
the five aspects of self under investigation. Table 1 summarizes the 
results, 


TABLE 1 


Correlation of Unsigned Self Reports and Inferred Self Concepts 
When Stories Were Held in Confidence. N = 30 


Aspect of Self Concept r 
1. The self 213 
2. The self in relation to others -140 
3. The self as achieving -166 
4. The self in school 312 
5. The physical self 1359 
Average Correlation 1245 
.05 level of significance “360 


There were generally low correlations with none exceeding the 
360 value required for the .05 level of significance. The average cor- 


elanon was ascertained by transforming each correlation to Fish- 
er's 2 value, averaging, 


was .245, a value si 


gned and unsigned self re- 


ports. The results are presented in Table 2. It is apparent that 


‘ 
{ 
É 
l 


1 a 
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there was a rather high degree of agreement between the self re- 
ports made under anonymous conditions and those made when the 
subjects were not promised anonymity. The implication is that the 
introduction of the social expectancy factor did not greatly influ- 
ence the subject’s responses on the self report. 


TABLE 2 
Correlation of Unsigned and Signed Self Reports. N = 30 
Aspect of Self Concept T 
1. The self .804 
2. The self in relation to others .735 
3. The self as achieving .718 
4. The self in school .652 
5. The physical self «763 
Average Correlation -740 
,01 level of significance 463 


When correlations were computed for self concepts inferred 
from stories told under conditions of confidentiality with those 
told when confidentiality was not guaranteed, the relationships 
were fairly high. Thus, the inferred self concept did not reflect 
change which could be attributed to the introduction of social ex- 
pectancy. The data is summarized in Table 3. 

TABLE 3 


Correlation of Inferred Self Concept when Responses Were Held in Confidence 
and when Responses Were Not Held in Confidence. N = 30 


Aspect of Self Concept r 

1. The self 775 
2. The self in relation to others 791 
3. The self as achieving .605 
4. The self in school .685 
5. The physical self .564 
Average Correlation .695 
.01 level of significance .463 


It was expected that correlations between signed self reports and 
self concepts inferred from picture tories not held confidential 
would be lower than correlations between unsigned self reports and 
inferred self concepts from picture stories which were held confi- 
dential. Table 4 presents the data. 

The 7’s under confidential conditions are all positive and above 
140; whereas four of the 5’s under non-confidential conditions are 
negative. This might lead one, on cursory examination, to conclude 
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TABLE 4 


elati Self Re and Inferred Self Concept when Confidentiality 
on De eke Was Not. N = 30 


r Under r Under 
Confidential Nonconfidential 
Aspect of Self Concept Conditions Conditions 

1. The self 213 —.099 
2, The self in relation to others -140 -034 
3. The self as achieving 166 — .037 
4. The self in school 312 —.019 
5. The physical self 359 —.117 
Average Correlation .245 — .046 


that there were pronounced differences between correlations for 
each individual aspect of the self concept. However, when these 
differences were tested there were no instances in which the critical 
ratio was as great as .196, the value needed for the .05 level of sig- 
nificance. When the average correlation for all five aspects of the 
self concept was calculated, through the r-to-z transformation, the 
difference between the two r’s (.245 and —.046) was significant at 
the .01 level. On the basis of these results, then, it is concluded 
that there was a significant difference between the self report and 
the inferred self concept when confidentiality was not promised. In 
addition, the average r under non-confidential conditions (—.046) 
was not significantly different from zero, hence it may also be said 
that the two sets of results do not have great relationship, 

It was anticipated that the teacher of the children who acted as 
subjects would more readily recognize individuals from their self 
concept as inferred by a trained observer than from their own self 
report. The significance of the difference between observed and ex- 
pected probability was tested (Walker and Ley, 1953). 

The teacher’s task necessitated thirty choices (N = 30). The- 


by chance, she would select, 


a r 


_ ><». 
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Conclusions 


The results of this research furnish the basis for several con- 
clusions. 

1. The data produced by the study lend support to the claim 
that the self report and inferred self concept do not furnish the 
same insight into the personality of individuals. Even when con- 
ditions are provided which should allow the greatest freedom from 
threat and maximum opportunity for frankness the two methods of 
observing people show little similarity. Therefore, the assertion 
that the self report and the self concept are different is substanti- 
ated by this research. 

2. While this research found the self report to be consistent even 
though social expectancy was introduced, there is no basis for con- 
cluding that the self report accurately reflects children’s percep- 
tions of self. The fact that some of the children’s experiences in 
school did not seem conducive to the development of the essentially 
positive views of self they reported leads to hesitancy in accepting 
the self report as a measure of the self concept. 

3. As anticipated, the inferred self concept was not greatly in- 
fluenced by social expectancy. There is some reason to contend 
that the inferred self concept provided more realistic insight into 
individual perceptions. First, the researcher’s ratings were some- 
what less positive than were the children’s self reports and in sev- 
eral cases the self concept ratings were more in line with what 
would be expected, according to perceptual theory, in the develop- 
ment of self perceptions in light of individual experiences as re- 
vealed by school records. Further, the teacher who knew the chil- 
dren through daily experience tended to support the ratings made 
from the inferred self concept as generally in agreement with her 
own view of individuals. The conclusion is that the inferred self 
concept represents a more accurate and realistic appraisal of chil- 
dren’s perceptions of self than do self reports. 

4, Correlations between self report and inferred self concept 
diminished when social expectancy was emphasized. It is ap- 
parent that external changes can cause differences in statistical 
relationship between the self report and inferred self concept. How- 
ever, the data does not lend itself to a clearcut conclusion about 
the reasons for these differences. 
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5. The teacher recognized individual children more often from 
ratings made by the researcher from inference than from the chil- 
dren’s own self ratings. This suggests that two adults having simi- 
lar basis for understanding children, but using quite different 
methods of studying behavior, are likely to arrive at fairly similar 
impressions of children. A related conclusion is that the picture 
story approach offers a means of gaining information about chil- 
dren in a short time which is as good as knowledge accumulated 
over a longer time through informal association with children. 


Summary 


This study inquired into the relationship of the self report and 
the inferred self concept first under conditions calculated to re- 
duce the effects of social expectancy, and again when social ex- 
pectancy was emphasized. The results show that both the self re- 
port and the inferred self concept remained relatively consistent 
under the varying conditions. However, when the correlations be- 
tween the self report and the inferred self concept before emphasiz- 
ing social expectancy were compared with correlations between the 
self report and the inferred self concept after emphasizing that 
factor, there was a decrease in the statistical relationship. This 
suggests that some change did result from the operation of social 
expectancy. 

A further test was made by asking the teacher of the pupils who 
acted as subjects to select blindly a self report or a rating based on 
the inferred self concept as more nearly agreeing with her own im- 
pressions of each child. The teacher chose the inferred self concept 
rating in 70 per cent of the cases, 
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IMPORTANT NOTICE TO AUTHORS 


In view of the tremendous advances that have been made in the 
adaptation of electronic computers and accounting machines to the 
processing of statistical data, sections of the Spring and Autumn is- 
sues of EDUCATIONAL AND PSYCHOLOGICAL MEASURE- 
MENT are devoted to the publication of such programs as are ap- 
propriate to psychometric procedures. Programs relevant to such 
problem areas as factor analysis, item analysis, multiple regression 
procedures, the estimation of the reliability and validity of tests, 
pattern and profile analysis, the analysis of variance and covariance, 
discriminant analysis, and test scoring will be considered. Cus- 
tomarily a program should be expected not to exceed six or eight 
printed pages. Manuscripts of four or fewer printed pages are pre- 
ferred. Hach manuscript will be carefully reviewed as to its suita- 
bility and accuracy of content, In Some instances an accepted paper 
may be returned to the author for possible revisions or shortening, 

Due to the financial pressure of rising printing costs, the new rate 
for publication of articles in this section, beginning with the first 
issue for the year of 1967, will be twenty-five dollars per page, The 
extra cost of the composition of tables and formulas will be added 
to the basic rate. 

Manuscripts received up to November first will be considered 
for the Spring issue; manuscripts received between then and May 
first will be considered for the Autumn issue. 

All correspondence and manuscripts should be directed to: 


William B. Michael 


Professor of Education and Psychology 
University of California, Santa Barbara 
Santa Barbara, California 93106 
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A GENERALIZED ONE-WAY ANALYSIS OF 
VARIANCE PROGRAM IN FORTRAN-II! 


ALLAN J. NASH 
Florida Atlantic University 


systems with maximum flexibility in the analysis of variance for 
single factor experiments. Its primary feature is a sense switch 

4 option to analyze differences among k treatment levels according to 
a model which assumes correlated observations (repeated measures 
or matched groups designs) or according to a model which assumes 
non-correlated observations (randomized groups designs). The sta- 
tistical models underlying these two basic analyses have been de- 
scribed by Winer (1962). 

The program was initially written for the IBM 1620; but since 
¢ it requires no special equipment or subroutines, it can be used on 
most systems with a FORTRAN-II compiler. The program itself 
requires approximately 8K core storage capacity. Since the com- 
plete data matrix is needed only when the correlated data option is 
exercised, it is possible to conserve core storage during a non-cor- 
related data analysis by cycling each observation individually 
through the preliminary summations with little cost of machine: 
time. This feature, in addition to the program’s ability to process 
unequal sample sizes, permits the analysis of any single-factor 
experiment which assumes uncorrelated observations, regardless of 
the design dimensions or additional core storage capacity. When 
correlated observations are analyzed, the complete data matrix 
(k treatments X n subjects) is required and the remaining core 
storage will determine the maximum permissible dimensions of the 


r. Tuis program was developed to provide intermediate computer 


1 The author is indebted to the Computing Center of Temple University 
for assistance in developing and testing this program. 


703 


704 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


design. The formula, 10 (kn + n + 3k), gives the number of 
additional cores needed for processing correlated data. 

Dimension statements must be changed in the program for dif- 
ferent values of k and n. It is also necessary to modify the data 
input formats if the card-column fields change from one analysis to 
another. The absence of a variable format capability thus necessi- 
tates recompilation for analyses with differing design or data for- 
mat characteristics. This disadvantage is balanced by the elimi- 
nation of the special subroutines and equipment needed to achieve 
variable format capability. 


Input 


I. Correlated Observations (sense switch OFF) 

1. Source deck with dimension and data format statements ap- 
propriate to the specific analysis. 

2. A parameter card defining values for k (number of treatments 
or classifications), n (number of subjects or blocks of cor- 
related observations), sample sizes for the k treatments (must 
be constant), and the total number of observations, 

3. The n data cards, in any order, each containing k correlated 
observations. 

II. Uncorrelated Observations (sense switch ON) 

1. Source deck with dimension and data format statements ap- 
propriate to the specific analysis. 

2. A parameter card defining values for k (number of treatments 
or classifications), n (a control constant set equal to 2), sam- 
ple sizes for the k treatments (need not be constant), and the 
total number of observations. 


3. The data cards, ordered by treatment level, each containing 
one observation. 


Output 

; In addition to treatment means, standard deviations, 

sizes, the following output is given: 

I. Correlated Observations 

1. Mean squares and degrees of freedom for between-subjects 
and within-subjects variation. This latter source of variation 
is further broken down to give Mean squares and degrees of 
freedom for treatment variation and the residual error term. 


and sample 


Phe 
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2. F-ratio: F = M Streatments/ M Sresiauat 
II. Uncorrelated Observations 
1. Mean squares and degrees of freedom for between treatment 
variation and within treatment (error) variation. 
2. F-ratio: F = MStreatments/MSerror 


REFERENCE 


Winer, B. J. Statistical Principles in Experimental Design. New 
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EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
1966, 26, 707-708. 


A NOTE ON A MODIFICATION OF COOLEY AND 
LOHNES’ CLASSIFICATION PROGRAM? 


GEORGE H. DUNTEMAN 
College of Health Related Professions 
University of Florida 


AurnoueH Cooley and Lohnes’ classification program (Cooley 
and Lohnes, 1962) was designed primarily to compute classification 
equations in the reduced discriminant function space, their program 
also contains provisions for obtaining quadratic (non-linear) dis- 
criminant equations in the original test space. The output for the 
classification program includes the group membership predictions 
based upon two basic quadratic discriminant equations. The first 
quadratic form is (X — X,)’ Dr (X — Xj) where (X — X;) is a 
vector of deviation scores from the test means of group 7 and D;* 
is the inverse of the dispersion (variance — covariance) matrix for 
group i. The second quadratic form results ina probability state- 
ment of group membership and is proportional to (X — X” Drt 
(X — X,) + log, |D;| — 2 log. P; where |D,| is the determinant of 
the dispersion matrix for group 7 and P; is the a priori probability 
of being a member of group i. 

The second equation is optimal in the sense of minimizing mis- 
classification. However, it assumes that the a priori probabilities of 
group membership are known. In most instances involving psy- 
chological and educational research, the number of people included 
in the analysis for each group is arbitrary (i.e., the number and 
proportions of people included in the various groups is a function of 
availability) and consequently the a prior: probabilities are mean- 


1 This research was supported in part by a research grant, #RD-1127, from 
the Vocational Rehabilitation Administration, Department of Health, Educa- 
tion, and Welfare, Washington, D. C. 
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ingless and their inclusion in the classification equation distorts the 
picture of classification efficiency. On the other hand, the first equa- 
tion does not utilize all the available information, since it assumes 
that the determinants of the group dispersion matrices are equal 
when in fact the ellipsoids in the multivariate space may not be of 
equal size. 

A more useful and general quadratic discriminator, utilizing dif- 
ferences among the dispersions of various groups but assuming equal 
a priori probabilities of group membership is (X — KIDA (X — 
Xi) + log, |D;|. This quadratic form degenerates to the linear form 
if the dispersion matrices are equal. This last equation may be 
easily obtained from Cooley and Lohnes’ program since a sub- 
routine called by the program has already calculated |D,| for inclu- 
sion in the Bayesian probability equation. An addition of a few 
Fortran statements easily results in the computation of log, |D| and 
its addition to the first equation already computed by the program. 


REFERENCE 


Cooley, W. and Lohnes, P. Multivariate Procedures for the Be- 
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A FORTRAN GENERATOR OF TEST SCORES 
ACCORDING TO A PREDETERMINED 
FACTOR PATTERN 


SIDNEY M. ROSENBLATT* 
Teachers College, Columbia University 


An interesting method of testing the truth of factor analysis is to 
factor analyse test scores which already have definite factors built 
into them. This method was first used by McNemar (1941) to test 
the sampling error of factor loadings. The test scores are determined 
by the linear factor model. 

X = fı (Rs) + fa(Bs) + fa(Bs) E+ + fa(Rn) + fa(Re) 
+ fey(Rex)- 

fr fay. ++) fr are the factor weights in the factor pattern. 

fais a term put in to account for specific variance. 

fa is a term put into the model to account for error variance. 

Ra, Rs, - ++, Rn are normalized random number weights generated 
by the computer. 

Xa Xe, Xs, ---, Xn are test scores. 

The program provides a punched card output of the test scores 
to facilitate factor analysis of the test scores. The input of the pro- 
gram is a control card describing the number of students and the 
number of tests for each student. For each test used in the battery 
a data card is prepared giving the factor weights, the error portion 
of the variance, and the specific portion of the variance. 

This program used a random number generator subroutine de- 
scribed in Gruenberger and McCracken (1963) and a subroutine 
which transforms the random number into a normalized random 

1This program was developed by the author in cooperation with Mr. Joel 
Herbsman, formerly of the Teachers College Computer Center and now of 
the University of Buffalo Computer Center. 
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number as determined by the equation for the normal probability 
distribution. 

The author generated three samples of twenty test batteries on 
the IBM 1620 II, through using this program. The test batteries, 
which contained from ten to twenty tests, were generated for 100 
students, The running time to generate twenty test batteries, five 
containing ten tests, five containing twelve tests, five containing 
sixteen tests, and five containing twenty tests, for 100 students was 
approximately one hour on the IBM 1620 TI. 

A program deck is available from the Teachers College, Colum- 
bia University Computer Center, filed under the author’s name. 
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A FORTRAN-IV PSYCHOLOGICAL TEST-SCORING 
PROGRAM 


SIDNEY R. ADELMAN anp WILLIAM H. McWHINNEY 
Graduate School of Business Administration 
University of California, Los Angeles 


Purpose 


Tus program analyzes multiple-response questionnaires in which 
the test score is a series of individual factor scores. It takes either 
a two-, three- or a five response questionnaire, and computes up to 
fifteen factor totals, A total factor score is the sum of the responses 
of the questions associated with or keyed for the factor. For every 
case, the program determines: (1) total factor scores, (2) the stand- 
ard deviation for each score, (3) the percentage of the total scores 
or the total number of items on each factor scale that are answered 
in the keyed direction and (4) the percentile value corresponding to 
the total score on each factor. Norms are developed for the entire 
sample, and if desired, an analysis of the norms by ages is also avail- 
able, It has been used to score the Buhler-Coleman Life Goals In- 
ventory (Buhler, 1963) and the Personal Orientation Inventory 
(Shostrom, 1963). 

The user has the option of computing Cronbach and Gleser’s 
(1953) D-square in which the sum of the squared differences is de- 
termined for each pair of cases. This matrix can be punched in a 
form which is accepted by Ward and Hook’s (1963) Hierarchial 
Grouping Program. Total factor scores as well as the correspond- 
ing percentage scores (mentioned in (1) and (3) respectively in the 
preceding paragraph) can also be punched at the user's discre- 
tion. 
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Program Specifications 
Control Cards 


A variable format statement is used to specify the form of each 
case input. The first control card specifies the number of cases, of 
questions, and of factors. This card is also used to specify the use 
of the options which are available. The second control card indi- 
cates the number of questions in each factor. The next set of cards 
indicates the question numbers for each factor (one card for each 
factor), If an inverse rating is to be given to a particular question, 
the question number is preceded by a minus sign. The next set of 
cards contains previously established norms. 

These norms will be compared against each case's total factor 
score (mentioned in (1) in the first paragraph). If these norms have 
not yet been determined, or are not desired, a blank card for each 
factor must still be included. 


Data Input 


Case input may be in the form of punched tabulating cards or 
magnetic tape. Each case is identified by a three- or four-digit 
code number, a required indication of sex, an optional age designa- 
tion, followed by the question scores. The program allows either 
two (1 or 2), three (1, 2 or 3), or five (1, 2, 3, 4 or 5) responses. It 
checks for invalid responses and when such are found, the code 
number as well as the offending question number is indicated. 
When no response has been made, either this outcome can be con- 
sidered to be invalid, or a three is assigned in the case of five pos- 
sible responses; a two is assigned when three responses are possible; 


and in the two response situation, a one alternating with a two is 
assigned, 


Capacity 


The program can now accommodate 400 cases, 160 questions and 
15 factors. By manipulating the DIMENSION statement, modi- 


fications can be made to handle programs with a variety of param- 
eters. 


wW 
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Running Time 


Rumning time depends on the number of options employed. When 
the scoring routine alone is used, the IBM 7094 uses on the order of 
one second for each case. 


Availability 


The program was written in FORTRAN-IV. It was compiled 
and “debugged” at the Western Data Processing Center at UCLA. 
Source decks, object decks, and appropriate documentation are 
available from the senior author. 


Summary 


This paper has described a general purpose scoring program 
geared predominantly for questionnaires. Its major attributes are 
its flexibility and ease of modification. 
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SCORING AND ANALYZING TEACHER-MADE TESTS 
WITH AN IBM 1620 


J. MICHAEL O’MALLEY 
San Jose City College 
AND 
CURT STAFFORD 
San Jose State College 


A recent article (Stafford and Bianchini, 1963) described a pro- 
gram by which teacher-made objective tests could be scored by an 
IBM 1620 after students had recorded answers on a special mark- 
sense answer card. The program was developed for the purpose of 
scoring and analyzing classroom tests in a fashion which previ- 
ously had been restricted to standardized tests. The output in- 
cluded scores on the total test; frequency distribution and T 
scores; the frequency and per cent with which each item alternative 
was selected (choice analysis); and a statistical analysis consisting 
of a count of the number of tests, the mean, standard deviation, 
standard error of measurement, and a KR-21 reliability estimate. 
Limitations imposed were: one response per item, a maximum of 
486 items and 999 examinees. The interest shown by local faculty 
and by faculty at other institutions indicates that the system has 
contributed significantly to the educational process. 


Two Revisions in Previous Program 


The purpose of this article is to describe two separate revisions 
of the first scoring program and to project possible future changes. 
The revision to be described first is known as SSS VI (Sure 
Swift Scoring, Revision VI). The major modifications are these: 
(a) provision of biserial correlations between item response and 
total score; (b) provision of up to four subscores, each permitted 
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a different scoring formula; (c) calculation of a KR-20 estimate 
of reliability, a more accurate estimate (Cronbach, 1962) of test 
reliability; (d) batch processing of consecutive sets of tests; (e) 
formating compatible with 80/80 listing; and (f) suppression of 
any combination of the four major program functions, (scoring, 
statistics, frequency distribution and T scores, and choice analysis 
with biserial correlations) by digit punch in the parameter card. 
There have been modifications of a rather minor nature in the for- 
mat of the parameter card and the output, but the new program 
still scores up to 486 items for 999 examinees. 

The second revision to be described, SSS VII, was developed be- 
cause of requests for Flanagan correlations by faculty not familiar 
with biserials. Initially, Flanagan correlations based on propor- 
tions correct in the upper and lower 27 per cent of the score distri- 
bution were provided through two separate programs, one for 
scoring and one for determining Flanagan coefficients. A two-pass 
procedure obviously was both cumbersome and time consuming. SSS 
VII scores tests and computes correlations in a single pass for a 
maximum of 150 examinees, 162 items, and 10 sets in a batch. These 
more restrictive limitations were imposed by the necessity of stor- 
ing both a large table of Flanagan correlations and the item re- 


Sponses of every examinee, In all other respects, however, SSS VII 
has the same flexibility as does SSS VI. 


Projected Modifications 


Programs developed to date have been for an IBM 1620 with 
40K core storage and with special features of indirect address and 
automatic divide. The recent addition of the 1311 disk drive 
greatly increases readily accessible storage. Immediate modifica- 
tions will be to expand the relatively narrow restrictions on the 
SSS VII (Flanagan) program, and to include T scores with the in- 
dividual examinee data. It would be highly desirable to add stu- 
dent name to the output, a fe: 


leges already have incorporated into their systems. Some faculty 


accumulation of test scores over the Semester; these will be investi- 
gated. 


The Testing Office, San Jose State College, solicits information 
from other classroom test scoring service Agencies, and conversely 
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is willing to share information and source or object decks from its 
library. 
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RELIABILITY AND VALIDITY OF THE DIGITEK 
OPTICAL SCANNER? IN TEST SCORING OPERATIONS 


RICHARD E. SPENCER 
University of Illinois 


Wiru the introduction of optical scanning equipment to the field 
of testing, it seems proper to determine degrees of reliability and 
validity of these relatively new and unique machine systems. Re- 
liability, in normal measurement terms, can be defined as relating 
to the degrees of consistency of the internal aspects of the parti- 
cular system. Reliability of a machine system can be computed 
analogously, as follows: 


1, Consistency between the printed and the punched test score. 
In image mode, the Digitek scores (i.e., counts) the number of 
correct marks, prints the test scores on the answer sheet, and 

_ punches a card with the score and items through the punch unit.’ 
The degree to which the printed score (on the answer sheet) agrees _ i 
with the punched score on the card is an indication of the internal 
reliability of the machine system. a, 


2. Consistency between the item read unit and the item punch unit. 

A second method to determine reliability is through the use of the 
item punches in the card output. The score printed on the answer 
sheet is determined by the optical read system in picking up the 
marks on the answer sheet. These marks are transferred into stor- 
age in a direct “image” of the answer sheets, and the memory is 
“dumped” into the card output. If one were to read these punched 
output cards as item responses, the test score developed by reading 
the punches should equal the test score punched in that same card, 
The degree to which such scores are equal can indicate the reliabil- 
ity of the machine system. g 


ee a 
1 Digitek Corporation, Fairless Hills, Pennsylvania. 
$ 719 


720 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


3. Consistency of the print unit. 

A third measure of internal reliability of the system can be de- 
termined by the degree to which the Digitek yields the same re- 
sults a second, third, or fourth time. 

The validity of the machine system, again defined in the meas- 
urement sense, can be defined only with the use of some outside 
criterion. In the case of a test scorer, validity can be determined by 
comparing the results obtained through the use of the Digitek with 
results determined by some other means. In this instance, hand 
scoring can be used to check the validity of the Digitek test scoring. 

These reliability and validity definitions can be summated as 
follows: 

Reliability #1: Score printed on answer sheet compared to score 

punched on output card. 

Reliability #2: Score punched on output card compared to score 

derived from item “image” punches. 

Reliability #3: Score of first machine pass compared to score on 

successive machine passes, 

Validity #1: Digitek derived scores compared to hand derived 

scores, 


Procedure 


Computation of reliabilities number 1 and 2 was performed on 
tests used in undergraduate foreign language programs, Spring se- 
mester 1964-65. Digitek answer sheets were used (University of 

„Pnois Answer Sheet MLA). The tests were the Modern Language 

“ Association, Cooperative Foreign Language Tests in French Read- 

ing and Listening Comprehension, (Form LB and MB). The ex- 

perimental sample was composed of students enrolled in French 

101, 102, 103, and 104 during the Spring semester at the University 

of Illinois. The following tabulation indicates the number of stu- 
dents in each sample: 


No.of No.of No. of 


Course Test Items hass 
ros pasita Hew oS Si 
Fren Listening LB 45 1 164 

ch 102 Reading LB 50 i 402 

Fr Listening LB 45 1 417 
ench 103 Reading MB 50 1 149 
Listening MB 40 1 151 

French 104 Reading MB 50 1 491 
Listening MB 40 1 491 
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There were two answer sheets per student; one for the reading 
test and one for the listening tests. The answer sheets were scored 
on the Digitek, and the test score printed on the answer sheet, 
punched on a card output, and the item responses were also punched 
out on the output card. A listing was made of the total score by 
student as punched in the output card, and clerks compared the 
listing with the printed score on the answer sheet, It was intended 
that a list of the two scores would be punched, the data submitted 
to a computer, and a correlation computed. However, since no dif- 
ferences were found, the correlation was not computed. The results 
indicated a perfect relationship between the two sets of cards 
(2444 score sets). Reliability #1 was 1.00. 

Reliability #2 was determined by submitting the Digitek item 
score cards to a computer item analyses/test scoring program with 
the appropriate key for each test. Four sample sets were devel- 
oped. Each punched test score was compared with the computer test 
score, and a correlation determined. The results were as follows: 


1, French MB Listening, N = 642, r = 1.00 
2. French MB Reading, N = 635, r = 1.00 

3. French LB Listening, N = 571, r = 1.00 

4, French LB Reading, N = 596, r = 1.00 


These results indicate a perfect relationship between the score 
developed by the optical read system, and the score determined by 
the punched item output. Reliability #2 was 1.00. 

Reliability #3 was determined on a different sample of stu- 
dents, A class of elementary biology (Biology 100) from the Chicago 
Campus of the University of Illinois comprised the sample, contain- 
ing 1123 students. The tests were packaged and sealed at the site of 
test administration, and opened upon receipt at the Digitek. The tests 
were in no way specially prepared for the Digitek run. The students 
used regular lead pencils provided by themselves, Three forms of 
the test were used, Form A, B, and C, each with 20 items. Each 
form was scored separately with the Digitek “select” switch on, and 
the “multiple response” switch turned to select. Each test paper 
was processed three times. The following results were noted: 


Form A, N = 284 
One paper selected out all three runs, because of one question 
being marked twice, and one question being marked three times. 


a 
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Six differences were noted as follows: 


. 15, 14, 15 
. 8, 12, 12 
. 4, 6, 6 
. 6, 12, 12 
. 5, 8,8 
6. 14, 16, 16 
Form B, N = 418 
Two papers were selected out all three times due to unclear 
erasures. 
No differences in test scores were noted. 
Form C, N = 421 
Eight papers were selected out all three times because of dou- 
ble and triple marks. 
One difference was noted as follows: 
1. 14, 14, 13 
Out of 284 papers in Form A, 6 differences occurred 
Out of 418 papers in Form B, no differences occurred 
Out of 421 papers in Form C, 1 difference occurred 
Out of 1123 papers, 116 were scored consistently three times. 
Seven papers had a difference. The degree of consistency or reli- 
ability can be determined by the proportion of equal scores on all 
three runs; 99.38 per cent of the scores were equal all three times. 
The validity of the system was checked on a second test given 
to the Biology 100 students from Chicago. On this separate oc- 
casion, 1086 tests were administered; again, three forms were used. 
Each test contained 20 test items. The tests were hand scored in 
order to obtain a validation criterion, Where differences occurred, 


the test was hand scored a second time, The following results were 
noted; 


orf Wh 


Form A; 1 error of 5 points 
Form B; 1 error of 1 point 
Form C; 3 errors of 1 point 


Five error papers were discovered out of 1086 papers, an overall 
accuracy rate of 99.54 per cent. 

In order to determine further the no: 
other test scoring systems, 
scored prior to being submi 


rmal accuracy expected by 
one set of 203 test papers was hand 
tted to the Digitek. After the Digitek 
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rescored the tests, a comparison was made, and again, differences 
were rescored by hand. Eleven differences were discovered. Upon 
rescoring, the Digitek scoring was found to be 100 per cent ac- 
curate, and the hand scoring (one time only) was found to be 
96.11 per cent accurate. 


Conclusion 


The results obtained in this study of the reliability and validity 
of the Digitek Test Scoring System would indicate that values in 
excess of r = .99 can normally be expected. 
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A MACHINE SCORING ANSWER SHEET FORM 
FOR THE IBM 1231 OPTICAL SCANNER 


JOHN A. FINGER, JR. 
Rhode Island College 


Oprican scanning scoring equipment has necessitated the re- 
design of answer sheets to meet the requirements of specific equip- 
ment. Unfortunately, there has been no standardization of equip- 
ment so that new forms are needed to suit the limitations of the 
scanners. 

Both the Digitek and the 1280 IBM scanners are able to handle 
only one answer sheet at a time. Thus test batteries necessitating 
use of more than one answer sheet require either manual card col- 
lating or large size computers capable of collating magnetic tape 
or disk record. Many test batteries (for example, Houghton Mif- 
flin’s, 1230 answer sheets for the Iowa Tests of Basic Skills or the 
Psychological Corporation’s Differential Aptitude Tests) have an- 
swer sheets requiring a student code number on every answer sheet. 
This requirement is time-consuming in test administration. More- 
over, miscoding of a student number is a major problem in col- 
lating punch card output. 

Test scoring equipment needs to be designed which can process 
all responses on a student’s answer sheets at a single pass. Such a 
procedure can now be accomplished through utilizing the Digitek 
or the IBM 1231 optical scanner when it is connected to any com- 
puter capable of accepting its output. 

Alterations are much needed in the design of both Digitek and 
IBM scanners. It should be possible with only minor design 
changes to provide multiple options with a scanner test scoring 
machine and computer as follows: 
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a. Separate test scoring without the computer. 

b. Scanner scores tests and provides identification information 
and scores to the computer. 

c. Scanner transmits bit by bit test information. 


The Rhode Island State Testing Program utilizes answer sheets 
in which the student's name is gridded and then decoded by com- 
puter. Multiple answer sheets are precoded with a printed scannable 
number to assure that a student’s complete record will be assem- 
bled. The computer (IBM 1440, 4K) which can accept scanner 
output for up to four answer sheets sequentially for one student, 
produces a single punch card containing the student name and 
scores, 

The top portion of the first answer sheet of the packet of four 
(attached with a 3/4 inch left margin peforated stub) is shown as 
Table 1. Other answer sheets contain only answer responses except 
for the preprinted scannable number. The first digit of the scan- 
nable number identifies whether it is answer sheet one, two, three or 
four. 
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DESIGNING AND PRINTING IBM 1230 OPTICAL 
MARK SCORING READER ANSWER SHEETS 
BY PHOTO-OFFSET 


JOHN F. GUGEL 
Hunter College 


Tum new IBM Optical Mark Scoring Reader can eliminate cum- 
bersome hand key punching jobs. This machine when used with its 
key punch attachment (the 534 Key Punch), is capable of quick 
and accurate automatic transfer of information from special an- 
swer/response sheets to IBM cards, These answer/response sheets 
are coded with ordinary number 2 pencils. However, when they are 
designed and supplied commercially, the cost is high and delivery 
is slow. 

Thanks to an attempt at printing the answer sheets by Max 
Ferder of Borough of Manhattan Community College, the writer 
has been able to develop a procedure for designing and printing 
them by photo-offset. Thus, not only costs can be reduced, but also 
individual code sheets can be provided for the specific demands of 
a single research project. 

Tt is specified that the answer sheets must be in two colors—the 
response positions in a reflective ink and the timing check marks in 
the right margin in a non-reflective ink (black). However, two- 
color printing is not really necessary. Answer sheets can be made 
by ordinary photo-offset if the black ink is kept light. The surest 
way to guard against dark printing is to have the printer use the 
process newspapers employ to reduce the blackness of a picture. In 
this process, black areas are reduced to areas of small dots, barely 
visible to the naked eye. When this process is used on the response 
section of an answer sheet, the black ink is reduced to a shade of 
gray and the timing marks remain a heavy shade of black. A hue 
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of the response section called “fifty per cent of black” by the 
printer is quite adequate. A hue as light as “ten per cent of black” 
is possible. 

Perfect operation of the machine’s timing mark check system is 
crucial in assuring accurate data transfer; therefore, great care 
must be taken to keep the distance from the edge of the sheet to the 
timing marks at 11/32 of an inch. It is a good idea for the printer 
to keep the timing marks on the opposite side of any edge to be 
cut. This can help cut down on top to bottom right margin fluctua- 
tions which cause trouble. Also, the distance from the top edge of 
the paper to the first timing mark should be 1/4 of an inch. If these 
specifications are not adhered to, the responses may be missed, and 
the processed data will be inaccurate. The best way to assure ac- 
curate measurements is to use the Document Inspection Gage which 
is usually kept in the side shelf of the IBM 1230. This instrument 
shows what the answer sheet tolerances are. 

With this photo-offset process, it is possible to construct re- 
sponse sheets to fit almost any test or questionnaire. For instance, 
in the preparation of a special response sheet for a demographic 
questionnaire administered to some twelve hundred Hunter Col- 
lege freshmen, a standard 0 to 9 response sheet was used. White 
paper was pasted over the areas to be deleted, and letters appli- 
cable to this particular questionnaire were typed on the sheet. 
Thus, for example, for sex of respondent two response positions 
were printed instead of the usual nine numbered positions (and 
0). If, on the other hand, it is necessary to insert additional po- 
sitions (within the limitations of machine design), response po- 
sitions may be lined up through the use of a mimeoscope and then 
pasted or traced onto the answer sheet. A newer design procedure 
which the author has found very useful involves starting off with 
a standard 1,000 answer position response sheet. A white opaquing 
liquid (the new kind used to remove errors in typing) is applied to 
remove all unneeded lettering, numbering, and answer positions. 
New lettering and numbering is then typed onto the answer sheet 
or written onto it with a very fine point pen. Then the answer sheet 
1s teady to be photographed for the photo-offset plate. 

Tt is also possible to print the original questionnaire on a response 
E E subjects often make stray marks on the question- 

are then read by the machine as extra responses. Also 
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lettering or lines crossing a response position or positions can 
cause problems if in black ink. Such an answer sheet cannot be used 
in programming the machine, although it may be used as a data 
sheet if the marking allows programming of the machine not to 
consider the answer positions crossed or covered. Thus, separate 
answer sheets are advisable. 

Many special purpose test and questionnaire answer sheets may 
be constructed with this photo-offset process. This process is not 
only a boon to the individual requirements of special research 
projects, but also an aid in reducing the cost of data processing. 
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AN IBM 1620 SPS COMPUTER PROGRAM FOR 
UNPACKING THE IBM 1230 SPECIAL CODE 


JOHN F. GUGEL 
Hunter College 


Wuen a standard 1 to 5 or A to E choice answer sheet is used in 
the IBM 1230 Optical Mark Scoring Reader—534 Key Punch 
System, questions are dealt with two at a time. The IBM 503 an- 
swer sheet, for example, has the first and second questions double 
punched by the 534 into one card column. Questions three and four 
are double punched into the next column, and so forth. When the 
534 drum card is programmed with a “3” punch for each of the 
card columns to be punched with information from the 1230, a 
special packed alphameric code results. Questions one and two 
become an alphameric character in one column; three and four be- 
come an alphameric character in the next column. 


The Program 


The purpose of this program is to translate the alphameric char- 
acters back into two columns. The type of answer sheet in which 
the choices are 1 to 5 or A to E then becomes represented by the 
punches 1 to 5 for each question. Question one is in one column; 
question two is in the next. Columns one to five of the input and 
output cards are reserved for an identification number. The identi- 
fication number should be punched into the card by using the “2” 
punch on the 534 drum card program. Part or all of the number 
may be plank. If more than seventy-five questions are packed into 
the card, Program Switch 1 of the 1620 must be turned on. When 
Program Switch 1 is left off, only one unpacked card will be 
punched by the 1622 for each packed card fed into the 1620. Two 
output cards are always punched when Program Switch 1 is on 
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during the processing. Both blanks and “12” punches are translated 
as blanks when they occur, 


1620 SPS Program 
DORG 492 
SF  INPUT+9,159,29 
AM FSET+6,1,16 
SM FSET+11,1,16 
BP FSET 
SF NB-2 
TF BB+11,NB 
TD TWOB+11,NB 
TD FOURB-+11,NB 
TD FIVEB+11,NB 
TD  ONEB+11,NB 
TD B4+19,NB 
TD B5+19,NB 
TD Bi+16,NB 
TD B2+19,NB 
TD B3+19,NB 
TD 3B+11,NB 
TFM TRANS+11,INPUT1+19 
TFM CFW+6,INPUT1-+9 
TFM TDW-+6,INPUTI+19 
RACDINPUT 
TR  INPUT1—1,INPUT-1 
BD COL4,INPUT1+7 
TD INPUTI+8,NB 
BD DIG4,INPUT1+5 
TD INPUTI-+7,NB 
B COL3 
TD INPUT1+7,INPUT1+6 
BD DIG3,INPUT1+43 
TD INPUTI+6,NB 
B COL2 
TD INPUT1+6,INPUTI-+-4 
BD DIG2,INPUT1+1 
TD INPUT1+5,NB 
B COLI 
TD INPUT1+5,INPUTI+2 
BD DIG1,INPUT1-1 
TD INPUT1+4,NB 
B WORK 
TD INPUTI+4,INPUTI 
BCI S1 
TFM WOP,38,19 
B TRANS 
at WOP,75,19 
AREA, INPUT1+19,7 
AM WAREA,11,19 A 
BP BB 
AM WAREA,3,1 
za FouR4 S 
WAREA,1,1 
BP FOURS” z 
AM 


WAREA,6,19 


= 
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BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 
AM 
BP 


TWOB 
WAREA,1,19 
FOURI 
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1g AM WAREA,1,19 
lil BP B2 

112 AM WAREA,1,16 
113 BP. B3 

114 AM WAREA,1,19 
115 BP 3B 

116 AM WAREA,1,19 
117 BP FIVE3 


118BB BTM CFW,9,9 
119FOUR4 BTM CFW,44,9 
129FOUR5 BTM CFW,45,9 
121TWOB BTM CFW,20,9 
122FOUR1 BTM CFW,41,9 
123FIVE4 BTM CFW,54,9 
124FIVE5 BTM CFW,55,9 
125THREE4BTM CFW,34,9 
126THREE5BTM CFW,35,9 
127THREEIBTM CFW,31,9 
128THREE2BTM CFW,32,9 
129TW04 BTM CFW,24,9 
139TWO5 BTM CFW,25,9 
131TWO1 BTM CFW,21,9 
132TW02 BTM CFW,22,9 
133TW03 BTM CFW,23,9 
134FOURB BTM CFW,49,9 
185THREESBTM OFW,33,9 
136FOUR2 BTM CFW,42,9 
1370NE4 BTM CFW,14,9 
1380NE5 BTM CFW,15,9 
1390NE1 BTM CFW,11,9 
1490NE2 BTM CFW,129 
1410NE3 BTM CFW,13,9 
142FIVEB BTM CFW,59,9 
143FOUR3 BTM CFW,43,9 
1440NEB BTM CFW,10,9 
145FIVE1 BTM CFW,51,9 
146FIVE2 BTM CFW,52,9 
1471B4 BTM CFW,4,9 
148B5 BTM CFW,5,9 
14981 BTM CFW,19 
1598B2 BTM CFW29 


151B3 BTM CFW,3,9 

1523B BTM CFW,39,9 

153FIVE3 BTM CFW,53,9 

154NOP NOP 

155CFW TD INPUTI+9,NOP+19,2 
156TDW TD INPUTI+19,NOP+11,2 
157 AM TRANS+11,2,1ø 

158 AM CFW+6,2,19 

159 AM TDW-+6,2,16 

169 SM WOP,1,16 

161 BP TRANS 

162 WNCDINPUTI1+4 

163 BNCISTART 

164 SF  INPUT1+4 

165 TF INPUT1+83,INPUTI+8 


166 CF INPUTI+79 
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167 CF INPUTI+4 
168 WNCDINPUT1+79 
169 B START 
17ØINPUT DAS 89 

171 DAC 1,@ 


172INPUT1 DAS 81 
173WOP DS 3 


176 DEND 492 


The speed of this program on the 1620 Model I is approximately 
forty-six cards per minute input when two output cards are punched 
for each input card. It is much faster when only one output card 
for one input card is desired. The identification number is always 
in both cards when two cards are punched for one. 

If it is desired to use this program as a subroutine, instructions 
6 to 17 inclusive may be eliminated and so forth. Blanks are then 
translated as zero punches. If an output identification number is not 
needed, instructions 23 to 40 inclusive and 175 may be eliminated. 
No 1620 special features hardware is required for this program. 
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PUNCHING MULTIRESPONSE QUESTIONS WITH 
THE IBM 1230 OPTICAL MARK SCORING READER: 
A PROCEDURE AND AN IBM 1620 SPS COMPUTER 

PROGRAM 


JOHN F. GUGEL 
Hunter College 


Unozr ordinary circumstances tests or questionnaires requiring 
more than one response at the same time to a given question can- 
not be punched by the IBM 1230 Optical Mark Scoring Reader— 
534 Key Punch System. The 1230 reads across the answer sheet by 
looking at ten response positions in a line on the left and by then 
following the same procedure on the right. When responses are to 
be punched for a test item analysis or for a questionnaire, the ma- 
chine does not punch less than the ten positions at one time. If 
more than one mark occurs in each set of ten positions, a multi- 
punch results. If there are more than three marks to the set, a “12” 
punch results when the 534 Key Punch is programmed with a 
“2” punch in the drum card for the column to be punched. The same 
outcome results if there are more than two when the “3” (al- 
phameric) punch is used on the drum. 

However, an answer sheet or questionnaire with more than one 
possible response per five choice question can be punched. As one 
faces the side of the IBM 1230 with the operating dials and 
switches, a panel door on the right side can be seen. When it is 
opened, a panel of twenty-one numbered buttons with inserts for a 
screw driver is visible. The buttons are used to change the bright- 
ness of the light bulbs that shoot light to the passing answer sheet 
for reflection back to the photoelectric cells. If the buttons are 
turned clockwise, the lights get brighter. If turned counterclock- 
wise, the lights get dimmer. (Before turning any of these buttons, 
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it is a good idea to mark where each button is set so that it can be 
reset.) If one of these buttons is turned completely counterclock- 
wise, its light goes out and the 1230 ignores all positions over which 
the bulb passes. For a set of ten bulbs it is possible to turn off nine 
and keep one on. The punching problem can be solved by making 
ten passes with each answer sheet, but the identification numbers of 
the papers would be split up onto several cards. 

When identification numbers and faster processing are required, 
two positions of the sets of ten may be taken at one time with the 
“3” punch on the 534 drum card used. This calls for five passes at 
most. Alphameric characters are punched for the two lighted 
marked positions. (One of the two marks must come from the first 
group of five question alternatives in the set of ten positions, and 
the second mark must come from the second group of five—two 
may not come from the same group of five at the same time.) 


The Program 


The following 1620 SPS computer program then can be used to 
put the identification number back together and combine desired 
combinations back onto one card: 


001 DORG 402 

0020RG BNI OFF,900 

0030FF TFM TR1+11,INPUT1—1 
004 TFM TR2+11,INPUT2—1 


005 TFM TR3+11,INPUT3—1 
006 TFM TR4+11,INPUT4—1 
007 TFM TR5+11,INPUT5—1 
008 RNCDHEADER 

009 SF HEADER 

010 MM HEADER-+1,2,10 
011 SM — 99,2,10 

012 A TR1+11,99 

013 A TR2+11,99 

014] A  TR3+11,99 

015 A — TR4+11,99 

016 A - TR5+11,99 
OI7START TFM A+6,0UTPUT-1 
018 TFM A+iLINPUT1-1 
019 TFM B+11,0UTPUT-1 
020 TFM C+6,0UTPUT-1 
021 TEM C©+11,INPUT2—1 
022 TFM D+11,0UTPUT—1 
028 TFM E+6,0UTPUT—1 
024 TFM E+1LINPUT3—1 
025 TFM F+11,0UTPUT-1 
026 TFM G+6,0UTPUT—1 
vag TFM G+1,INPUT4—i 
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028 TFM H+11,0UTPUT-1 
029 TFM I1+6,0UTPUT-—1 
030 TFM I+11,INPUT5—1 
031 TFM CUB,9,10 

032 RACDINPUTL 

033 RACDINPUT2 

034 RACDINPUTS3 

035 RACDINPUT4 

036 RACDINPUT5S 


037TR1 TR  OUTPUT+9,INPUTI-1,7 

038TR2 TR OUTPUT+39,INPUT2—1,7 
039TR3 TR OUTPUT+69,INPUT3—1,7 
040TR4 TR OUTPUT+99,INPUT4—1,7 
O41TR5 TR OUTPUT+129,INPUT5—1,7 


042A TD OUTPUT-1,INPUT1-1,2 
043B BD MOD,OUTPUT-1,7 
0440 TD OUTPUT—1,INPUT2-1,2 
045D BD MOD,OUTPUT-1,7 
046E TD OUTPUT-—1,INPUT3-1,2 
047F BD MOD,OUTPUT-1,7 
048G TD OUTPUT-—1,INPUT4-1,2 
049H BD MOD,OUTPUT-1,7 
0501 TD OUTPUT—1,INPUT5—1,2 
O5IMOD AM A+6,1,10 
052 ‘A+11,1,10 
053 AM B-+11,1,10 
054 AM C+46,1,10 
055 AM O©-+11,1,10 
056 AM D-+11,1,10 
057 AM E+6,1,10 
058 AM B+411,1,10 
059 AM F-+11,1,10 
060 AM G+6,1,10 
061 AM G-+11,1,10 
062 AM H-+11,1,10 
063 AM 1+6,1,10 
064 AM 1+11,1,10 
065 SM _ CUB,1,10 
066 BNF A,CUB 
067 WACDOUTPUT 
068 BNLCSTART 
069 RCTY 
070 WATYHDR 
1 B ORG 


O75SPACE DAS 80 


80 
079R2 DAC 1,@ 
80 
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085R5 DAC 1,@ 
086HDR DAC 17,LOAD NEW HEADER.@ 
087 DENDORG 


Columns 1 to 5 of the input and output are reserved for the identi- 
fication number. The output Tesponses are in groups of fifteen in 
columns 6 to 80. Input is transferred in groups of 15 columns 
from each card. The first 15 come from the first card, the next from 
the second card, and so forth. A header card precedes the data 
cards. It tells the computer from which column the input transfer is 
to start. This column is punched in columns 1 and 2 of the header, 
If, for example, the groups of 15 start in column 6, an “06” is 
punched in the first two columns, Then columns 6 to 20 of the first 
data card will go into columns 6 to 20 of the output card; columns 
6 to 20 of the second card will go into columns 21 to 35 of the out- 
put card; columns 6 to 20 of card 3 will go into columns 36 to 50 
of the output card, and so forth. Data input card one is followed 
by input card two, which is followed by input card three, and so 
forth. The input cards thus have to be carefully put in order and 
stacked together by hand. 

The author’s program for unpacking the special code should 
now be used. It is recommended that the light pairs set on the 
1230 be “1-1”, “2-2”, “3-3”, “4-4” up to “5-5.” Then the unpacked 
cards will have all the 1’s from the first cluster of questions fol- 
lowed by all the 2’s and so forth. 
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SCORING MULTIRESPONSE QUESTIONS WITH THE 
IBM 1230 OPTICAL MARK SCORING READER 


JOHN F. GUGEL 
Hunter College 


Unner ordinary circumstances tests requiring more than one re- 
sponse at the same time to a given question cannot be scored by 
the IBM 1230 Optical Mark Scoring Reader. The 1230 reads across 
the answer sheet by looking at ten response positions in a line on 
the left and then by doing the same on the right. In the scoring 
process the machine will score as many as four responses per ten 
positions if the Question Length 2 setting is used. Response po- 
sitions three and eight, however, would not be read, and papers 
with more than one mark per set of two questions would be selected 
and removed from the other papers. 

Nonetheless, questions with several right answers at the same 
time can be scored if an answer sheet is designed with lettering 
and numbering going down the page. With one position of each set 
of two across left blank, the Question Length 2 setting could be 
used. The author used this procedure in designing the answer sheet 
in this article from an IBM answer sheet by photo-offset. It was 
designed for use with the Differential Aptitude Test Battery. 

The special answer sheet is scored with the Question Length 2 
setting. It must be passed through a second time for a count of 
wrong answers (if needed). If the full number of questions is used, 
the number of right answers may be printed on one spot in the 
right hand margin and the number of wrong answers on the spot 
next to it. A little peg under the inner top hood of the machine 
makes it possible to print scores in two different places. 

If punching of questions is required as for an item analysis, the 
procedure should be used that is described in the author’s preceding 
article entitled “Punching Multiresponse Questions With the IBM 
1230 Optical Mark Scoring Reader: a Procedure and an IBM 
4620 SPS Computer Program.” 
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AN ALTERNATE TIME-SAVING PROCEDURE FOR 
COMPUTING Z SCORES? 


R. J. RANKIN 
Oklahoma State University 


Heaurn (1965) reports a fast method for computing z scores 
suitable for the two keyboard Marchant. The basic z score equa- 
tion is, 


xr 


z= (10) + 50, 


g 

where X, is an observed score, X is the mean of the observed 
scores, and o is the calculated standard deviation of the distribu- 
tion of observed scores. All short method z score computations are 
derived from the obvious linear relationship between raw and z 
scores, Health’s method requires computation of a constant 10/0 
and multiplying the raw score difference between a known z score’s 
equivalent raw score and the raw score for which z is desired, then 
adding the result to the last computed z score to obtain the new 
z score. This continuous process requires a running account of the 
distance between adjacent scores. The check requires the computa- 
tion of the mean of the obtained z scores. 

A simpler procedure, suitable for all one keyboard calculators 
and illustrated with a Monroe model IQ 213 desk calculator, does 
not require the computation of raw score differences by the opera- 
tor. 

1. Compute the z score for the highest (z1), and one digit less 
than the highest raw score (z2). z2 may not be equivalent to any 


— 


1 This project was supported by the Research Foundation, Oklahoma State 
University, Stillwater, Oklahoma. 
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obtained raw score, but the computed z’s must be adjacent in raw 
score terms. ; 

2, Find the difference between z; and z2, which is a constant (k) 
for any adjacent scores. f 

3. Clear the machine and lock the lower dials. 

4. At the far left of the keyboard, enter the highest raw score. 
Skip one column and enter zı, computed to the desired accuracy. — 

5. Enter the above from the keyboard to the lower dials. 

6. Under the one’s place for the raw score, enter 1. At the ap- 
propriate decimal place under the z, score, enter the constant (k) 
obtained in Step 2. f 

7. Make sure the keyboard will not clear automatically by can- _ 
celling the “automatic keyboard clear” and lock down the repeat 
slide into the repeat position. 4 

8. Punch the minus bar. The left position contains the raw score 
equivalent for z2, the right position shows z3. 

9. Punch the minus bar to obtain all desired z scores. 

10. Check by noting whether the z equivalent for the mean is 
equal to 50. 

With the exception of computing 21, z2, and k, the entire process — 
can be completed on the machine with no further computation, It 
is desirable to arrange the test papers in raw score order, but if one ~ 
is out of order it is a simple matter to run the + bar or — bar to © 
the desired raw score. 

Purists may wish to compute k to 7 places, but this is not nec- ~ 
essary unless the range of raw scores is wide. A good check on k is i 
to compute a z2 — zs difference which should be equal to z1 — 22, Í 

Statistios other than z may be computed with this procedure by ' 
simply changing the appropriate elements in the formula. 1 

No claim is made for the originality of this method, and the 
writer is certain that many others faced with the need for com- 
puting many 2’s have equivalent procedures. 


REFERENCES 


Health, H. A. Time-saving Procedure fi i 
‘or Computing Z Scores. 
ooh AND PSYCHOLOGICAL MEASUREMENT, 1965, 25, 323- 


Epucationat AND PsycHotocrcat Mi 
1966, 26, 749-811. Š TRAK eae 


BOOK REVIEWS 


WILLIAM B. MICHAEL, Editor 
University of California, Santa Barbara 
JOAN J. BJELKE, Assistant Editor 
Centinela Valley Union High School District 
and the 
University of Southern California 


Standards for Educational and Psychological Tests and Man- 
uals. Jonn E. Dossin, Roosme Lennon, Joun E, MILHoL- 
LAND, Kenner D. Hopxins, PHILIP HIMELSTEIN, JERRY C. 
Garrock, J. STANLEY AHMANN, AND MARY L. TENOPYR ... 751 


Lado’s Language Testing: The Construction and Use of For- 
eign Language Tests. SUZANNE STAHL ..esesosoesoreeene 767 


Stricker’s Acquiescence and Social Desirability Response 
Styles, Item Characteristics, and Conformity. ALBERT SIL- 


VHRSTHIN cc cccccececcecccscvcccecravsvoneccssvessrees 770 
Harrower’s Psychodiagnostic Testing: An Empirical Approach. 
PHILIP HIMBLSTHIN ...ssesreereereeeeeererererrererer. 772 
Downie and Heaths Basic Statistical M ethods. Rosurt R. 
IMORMAN amne ods eee moe Aa aaa esale enri Snae asa 774 
Mood and Graybill’s Introduction to the Theory of Statistics. 
Grorcs H. DUNTEMAN Hae HOM 
779 


Hadley’s Linear Algebra. Grorcn H. DUNTEMAN .. 
Goodlad, Caffrey, O’Toole, Tyler, Converse, and Thayer’s Ap- 

plications of Electronic Data Processing M ethods in Educa- 

tion. RICHARD WOLF ..--++eeeecsesteeucccesrerceeecers 781 
Harris’ Fortran Programming (II and IV). W. L. Basuaw .. 788 
Harris’ Numerical Methods Using Fortran. W. L. BasHaw .. 784 
Morrison’s The Proceedings of the Ninth College and Univer- 

sity Machine Records Conference. Ricuarp E. SPENCER ... 785 


749 


750 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
Lavin’s The Prediction of Academic Success. Joun C. Gowan 


Berdie and Hood’s Decisions for Tomorrow, Plans of High 
School Seniors After Graduation. C. RUSSELL pp Burro, Jr. 


Cervantes’ The Dropout: Causes and Cures. Norman M. 
etopo o cA coe cca tan BG Sey 


Breer and Locke’s Task Experience as a Source of Attitudes. 
Samiaa a 


Leavitt’s The Social Science of Organizations: Four Perspec- 
tives. EDWARD LEVONIAN .........0.0..0.0eseccces-s.s., 


Eysenck and Rachman’s The Causes and Cures of Neurosis. 
COON T T 


Thomas and Thomas’ Individual Differences in the Classroom. 


SeenON L.. 


Sears, Rau, and Alpert’s Identification in Child Rearing. Ger- 
ALD T. Kowrrz 


Pareck’s Developmental Patterns in Reactions to Frustration. 
Pum Hiara e ecseecss ss... 


Singer’s Substrata-factor Reorganization Accompanying De- 
velopment in Speed and Power of Reading at the Elementary 
School Level. Sara W, LUNDSTEEN 

Kolson and Kalu 
PATRICK Grorr 


ger’s Clinical Aspects of Remedial Reading. 


786 


788 


790 


792 


795 


797 


800 


802 


805 


| 
| 
| 
| 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
1966, 26, 751-811. 


Standards for Educational and Psychological Tests and M anuals, 
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The Professional Service Representative of a Test Publisher 


It is the opinion of one test developer and publisher (this one) 
that Standards may well be one of the half dozen most important 
publications in the history of measurement. Written in clear, con- 
cise English that is a tribute to the skill of its several authors, it 
enunciates fully a set of principles which have been growing for 
fifty years among the professionals of testing. 

This booklet’s two predecessor publications—Technical Recom- 
mendations for Psychological Tests and Diagnostic Techniques 
published by the American Psychological Association (APA) in 
1954 and Technical Recommendations for Achievement Tests pub- 
lished by the American Educational Research Association (AERA) 
and the National Council on Measurements Used in Education in 
1955 (now called the National Council on Measurement in Edu- 
cation [NCME])—represented a first and very large step toward 
codification of professional expectations for published tests. Stand- 
ards is“ second and even more impressive stride in the same 
direction. 

The book opens with an excellent description of its own history 
and purposes and methods, of the audiences who may utilize it, of 
the cautions to be observed by those who do use it—a modest and 
wholly objective set of directions for the reader. Then it goes on to 
present the “standards” in well-organized sections devoted to (a) 
dissemination of information, (b) interpretation, (c) validity, (d) 
reliability, (e) administration and scoring, and (f) scales and 


[Editor’s note: Shortly before the Standards for Educational and Psycholog- 
tcal Tests and Manuals was released, the editor sought reviews from several 
individuals with different professional orientations. At the beginning of each 
review is a caption or heading that describes the editor’s categorization of the 
professional identification of each person who on extremely short notice con- 
tributed a review of the Standards. Deep appreciation is expressed to each 
reviewer who participated in evaluating the Standards.] 
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‘norms. The clarity and comprehensiveness of its coverage of criti- 
cal topics are sure to gain wide use for this publication in the 
teaching of measurement as well as in the technical assessment of 
standardized tests. 

Not all publishers will agree that all the standards are appropri= 
ate, of course. Perhaps not even one publisher will think that 
every recommendation is both important and well-stated. B 
nearly all will agree that on every important point the commit 
has taken a position that is shared by substantial numbers 
specialists in the field—a position that can be defended with, vigor. 
This means that the committee actually is speaking for the p. 
fession by reflecting its divisions of opinion as well as its more 
nearly unanimous points of view, instead of legislating its owm 
rules for the publishers. So a listing of the specific points upon 
which the reviewer holds an opinion differing from that of the com- 
mittee would amount to little more than an interesting exercise; & 
different reviewer would produce a different list—and have am 
equally hard time proving that his position on a given point is 
better than the committee’s. ji 

As one would expect in a publication developed by a committee 
(even when the final writing is done by one or two very competent 
people) there is some unevenness in technical sophistication, as We 
as in expository and editorial styles, from section to section. Com= 
pared with the concepts and procedures specified in the section on 
reliability, for example, the recommendations pertaining to scales 
and norms are more often unspecific and occasionally over-sim= 
plified. And the section on scales and norms would be better with an 
introduction as useful and well-written as the introductions to the 
sections on validity and reliability. But this is strictly a compara- 
tive criticism, for all sections of the book are admirably well done, 
just as they are. “1 

Continuing the trend which was accelerated by its predecessor 
pabticnone, this book will make things even more difficult for the 
ae ie E as a publisher of operational tests. Aue” 
of meeting hes stand saa deans ot Aa tga re re 
LEM ME be! ty o n ee at the point of publication; by the time ' 
ae bee omplete most of the essential standards, the con- 

of their tests will have become obsolete. In this particular 
effect upon publishi i ey 
sélection ay shing practice, Standards offers a fairly accurate 
publishing, eg nel circumstances that influence” 
thet te teil nae al ; ne that satisfy current require- 
technical resources. eee een enor 

Standards should serve its central purpose w ime this 
review D surely, the major dai ae oon fa the 
recommendations by heart and will be planning to include in fu- 
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ture instruments those elements or standards which they have not 
already built into their publications. Further, measurement tech- 
nicians who buy and use tests will very quickly build Standards 
into assessment procedures for test materials. Finally, it ought to 
become evident to authors and teachers in the field that this publi- 
cation is an admirable statement of proper usage and nomencla- 
ture in the language of testing—amounting almost to a manual of 
style—which should be imitated widely. 

In conclusion, any reviewer whose career is related to test pub- 
lishing should be permitted one small, wistful hope. Partly as a 
consequence of the efforts of professional groups like APA, AERA, 
and NCME the technical standards of test publication have im- 
proved tremendously in the last two decades. It would be fair to 
say, however, that as a result of such general improvement a great 
many tests and manuals now are far better than the uses to which 
they often are put. A highly sophisticated instrument works no 
better than a cruder tool in the hands of an apprentice just learning 
the trade. To put high-standard instruments more often to high- 
standard uses, then, the technical qualifications of those who choose, 
administer, and interpret tests should be brought to the technical 
` level of the best tests they employ. Because it is evident that nearly 
all the recommendations in Standards need only the slightest turn- 
ing or re-phrasing to be wonderfully applicable as standards for 
the users of tests, and because no other group could possibly muster 
a comparable combination of professional influence and skill, it is 
to be hoped that the combined committees on test standards will 
undertake next the development of just such a publication; and 
make it as good as this one is. 

Joun E. Dossin 
Educational Testing Service 


A Measurement Specialist and Executive of a Large Test Publish- 

ing Firm 

Publication of Standards for Educational and Psychological 
Tests and Manuals, the up-dated version of the Technical Recom- 
mendations for Psychological Tests and Diagnostic Techniques, 
and for Achievement Tests, prompts this test publisher to ask him- 
self several questions: What effects have the Technical Recom- 
mendations had on test publishing in the decade since their ap- 
pearance? What effects have they had on test selection, interpre- 
tation, and use? Are the new Standards likely to be better or worse 
in any of these respects than the original standards? 

One approach to the first question would be to compare the 
manuals for tests published by the major publishers during, say, 
the past three years with the manuals of tests published in the 
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three years prior to issuance of the Recommendations. While the 
reviewer has not gone through this exercise, he does not hesitate to 
assert that the quality of test manuals has, indeed, improved over 
the past decade, nor to attribute much, perhaps most, of this im- 
provement to publication of the Standards. Other influences for 
good have, to be sure, been operative: the Mental Measurements 


Yearbooks, the general increase in level of sophistication concern- © 


ing tests, even, paradoxically, the wave of criticism of testing, 
which has forced the test makers to be on their mettle; but there is 
no gainsaying the impact of the Recommendations. 

To the reviewer’s knowledge, the Recommendations have been 
very much in the minds of the staffs of the major test publishers, in 
whose hands lies the responsibility for the preparation of most test 
manuals. They have perceived the Recommendations as a helpful 
codification of what the profession thinks it reasonable to expect 
test makers to provide in the way of information about their in- 
struments. “Conscience,” someone has written, “is a small voice 
telling us what we know.” The Standards may tell the responsible 
test maker little he does not, or should not know, but it is not a 
bad thing that he be told, in some organized authoritative way, 
what his fellow professionals expect of him. Most publishers, the 
Teviewer would judge, view the Standards (even those classed as 
“Essential”) as definitions of ideal practice rather than rigid pre- 
scriptions. Few, if any, manuals, even among those published in 
recent years, conform to the letter of the Standards in every re- 
spect; but it is a fair presumption that where a manual does not 
accord in every particular with the Standards, it is not because the 
Standards were ignored, but rather because a decision was taken, 
for good Teasons or bad, not to abide by the Recommendations in 
the particular instance. This reviewer has sensed no disposition on 
the part of any major publisher to dismiss the Standards as un- 
teasonable, unattainable, or irrelevant; indeed, it is hard to imagine 
that any serious test author or publisher would lightly disregard & 
body of recommendations developed with such obvious thought 
and care, and enjoying the endorsement of the most directly in- 
neler zal groups. 

18, OI course, not to be expected that every recommendation, oF 
the classification of recommendations as Weseential,” “yery desir- 
able,” or “desirable,” will command universal endorsement. This 
N er, for example, does not agree totally with the formulations 
E Standards on the use of correction-for-guessing 
EFE provision of tables of equivalence between new and re- 

rms as a universal practice, on handling the matter of local 


norms, on treatment of test scores over tim isi 
r b test e, on provision of slope 
ne information in connection with ail validity coeffi- 
; Ae shares the committee’s bias in favor of a standard-score 
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system of reporting and interpreting, but can sympathize with 
competing views, and so on. Every practitioner will have his own 
list of places where his judgment differs to some extent from that 
embodied in the Standards. Such lack of total agreement seems to 
the reviewer unimportant, and perhaps an inevitable consequence of 
the less-than-fully-developed state of our science. What is im- 
portant is that the reader sense the goal the committee had in mind in 
calling for certain kinds of information, and be guided by the 
spirit of the recommendations. 

Beyond the obvious use of the Standards as a guide in the prep- 
aration of test manuals, this publisher, at least, has found them 
helpful both with respect to the evaluation of test manuscripts be- 
ing considered for publication, and in the training of editorial staff. 

Less easy to answer are the questions on the effect of the Recom- 
mendations on test selection, use, and interpretation. There is little 
evidence to lead the reviewer to believe that many test users are 
now influenced in their selection of tests by the extent to which 
their manuals conform to the Technical Recommendations. Neither 
does there appear to be convincing evidence of correlation between 
improvement in test manuals and improvement in test use. Does 
this argue against the usefulness of the Standards, or call into 
question the wisdom of the sponsoring bodies in devoting so much 
time and effort to their production? The reviewer thinks not. He 
concludes that what is called for is much better dissemination of 
the Standards, and better training of test users in the application 
of the information which, thanks to the Standards, is increasingly 
available in test manuals. It is unrealistic to pretend that the 
modal user of educational or psychological tests today is behaving 
very differently from his counterpart of a decade ago as a result of 
publication of the Standards, and perhaps the Standards-writing 
groups never expected such an outcome. But there are grounds for 
some small optimism: most measurement textbooks that have ap- 
peared in the past five years have devoted attention to the Stand- 
ards, which attention, it is to be hoped, will result in improved 
test interpretation and use. 

How do the revised Standards compare with the earlier ones? 
In structure, organization, scope, and general tone, the revised 
Standards are very similar to the original Recommendations—in- 
deed, surprisingly similar in view of the amount of time and study 
devoted to preparation of the revision. Insofar as the Standards 
constitute a kind of index of the state of the science and the art, 
one is impressed at how little things seem to have changed in the 
ten years between the two editions. The treatment of validity, 
while somewhat expanded and recast, adheres to the concepts of 
content, construct, predictive and concurrent validity elaborated 
in the original Recommendations, although predictive and con- 
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current validity have been combined into one category. Most not- 
able changes occur in the section on reliability; here the committee 
has espoused wholeheartedly an analysis-of-error-variance ap- 
proach, a development which this reviewer counts as a decided 
improvement. Recommendations in the areas of administration and 
scoring, and scales and norms, are much like the original. 

Whether or not the revised Standards will be more efficacious in 
improving the quality of test manuals and level of test usage de- 
pends less on the changes between the new and the old editions 
than it does upon what steps are taken to bring the new Standards 
to the attention of test makers and test users. As far as the test 
makers are concerned, it is safe to assume that the new Standards 
will receive as much attention and observation as did the Recom- 
mendations. The real task continues to be that of consumer edu- 
cation aimed at increasing the user’s ability to utilize the in- 
formation that the test publishers are being stimulated to provide. 

A final word seems in order. To conform fully to the spirit of the 
Standards imposes formidable obligations on the test maker. The 
development and dissemination of all the information called for is 
a costly process, and often implies a long period of pre-publication 
experimentation. If the conscientious publisher is willing to under- 
take these additional developmental costs, and seeks to recover 
them, as indeed he must, in the form of higher prices for materials, 
he would like to feel that he will not thereby be placed at a com- 
petitive disadvantage and that the test user, in turn, will recognize 
an obligation to support better standards of test development by 
not preferring materials simply because they are less expensive. 

Test makers and test users, APA and AERA, are indebted to 
the committees that labored so hard both on the original Recom- 
mendations and on the new Standards. Their task has been 4 
difficult and perhaps a frustrating one, since in the nature of things 
the outcomes of their labors are so hard to assess. This reviewer 
judges that they have discharged their duties with sophistication, 
technical competence, and good sense. And if the Standards now 
and again seem to set goals that are unrealistic in a world of time 
and money, well, what is wrong with a dash of idealism? 


Rocer LENNON 
Harcourt, Brace, and World 
Test Division 


The Professor of Psychology 


After comparing the 1966 Standa i 
: ; : rds with the 1954 Recommen- 
oe ti reviewer gained the impression that there are not many 
ajor changes. The 1966 Committee worked long and hard. Thus it 
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is a tribute to the perspicacity of the 1954 group that their work has 
lasted so well. 

The new publication is, however, a little firmer in tone. One 
facet of this attitude is the change in title from Recommendations 
to Standards. Another example is the fact that in the 1954 Recom- 
mendations there appeared, in a section on Revision and Extension, 
the statement “The recommendations are intended to be used with- 
out reference to any enforcement machinery.” No such statement 
appears in the 1966 Standards. Still another instance appears in 
Item A1.22, which introduces the idea that promotional material 
for a test should also not be misleading. Likewise, Item A2.31, con- 
cerning the necessity for new data when a short form of a test is 
brought out, has been raised from Very Desirable to Essential. 

‘A new feature is a topical index, and this should be very useful. 
It appears, too, that it would have been very helpful for the Com- 
mittee to have prepared one or more outlines for the organization 
of a test manual, with citations of the paragraphs of the Standards 
that are relevant to each item. The reviewer had the feeling that 
standards bearing on a particular portion of a test manual were 
somewhat scattered. A confirmation of this impression is found in 
the number of different sections listed under some of the headings 
in the index. 

One prominent modification is in the discussion of validity, 
where the terms predictive and concurrent validity have been com- 
bined and labeled criterion-related validity. The appropriateness 
of content validity for achievement tests comes in for its proper 
emphasis, and the discussion of construct validity has been con- 
siderably clarified. Apparently twelve years of discussion and con- 
troversy about this latter type has borne some fruit. 

The section on Reliability has become more sophisticated, with 
advocacy of analysis of error variance as the most informative ap- 
proach. Reliability is defined as accuracy, and the reviewer must 
confess the Committee has not brought him along with them on 
this point. It seems that a test can be reliable but biased. 

One notes with approbation that the audience to whom the 
Standards are directed has been upgraded from competence at the 
level of a single course in tests and measurements to one of two or 
three such courses, plus two semesters of statistics. This explicit 
recognition that not just anyone is capable of judging the merit of 
a test is a healthy sign. 

The reviewer hopes that the shift in these Standards toward 
more rigor will stimulate the testing fraternity also to move to- 
ward a more stringent exercise of concern over what appears on the 
market, There has been reliance upon voluntary cooperation for a 
long time and, although there has been some progress, sometimes 
the most ethical of the test producers have been the ones to suffer. 
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The necessity for some kind of sanctions seems to be indicated 


both by the current importance of testing in our society and by the 
criticisms which have grown out of, and fed upon, legitimate pub- 


lic interest in what is going on. Perhaps a first step could be the | 
establishment, by the three bodies issuing these Standards, of a | 


“Seal of Approval” for test manuals! 


JoHN E. MILHOLLAND 
University of Michigan 


The Professor of Educational Measurement 


This brief set of guidelines for published assessment devices, like | 


its predecessors, is certain to have a marked impact in the testing 


field. The document is a well integrated and substantially improved | 


revision of two earlier independent, and partially overlapping, ef- 


forts by APA and AERA-NCME committees. It is clearly written © 


in language that places minimal demand on the reader’s measure- 


ment background. The tone of the Standards is suggestive, not dic- 1 


tatorial; care is taken neither to dictate practice nor to discourage 
change and innovation. The principal request throughout the vol- 
ume is for an honest portrayal of known information about an 
instrument. The reader will observe that most of the issues raised 
in the review are of minor consequence, often relating to matters 
of personal preference. 

_ The format employed closely parallels that of the earlier ver- 
sions, categorizing the recommendations into six divisions. The 
initial section on “Dissemination of Information” presents excellent 
guidelines for promotional literature on published tests. The sec- 
ond section, “Interpretation,” makes helpful recommendations re- 
garding information that should be included in test manuals to 
assist users in making correct interpretations of the test’s results, 
€g., making the distinction between statistical and practical sig- 
nificance explicit, 

The “Validity” section is substantially modified from previous 
versions. The three primary validity types, content, criterion-re- 
lated, and construct, are presented in a unified framework in such & 
way that it is clear to the reader 
typically involves information 
Process matric 


the term “item discrimination” “a sditv” for 
item-total score relationships is ty 


to be commended. The reviewer 
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would like to have seen some suggested guidance regarding the use 
of particular item discrimination indexes to assist in reducing the 
undesirable heterogeneity in current practice. 

Although the need for cross-validation is clearly indicated, es- 
pecially when multiple predictors are involved with small samples, 
the recommendation to apply a correction for shrinkage to multiple 
correlations when they are to be presented as evidence of criterion- 
related validity would seem to have been appropriate. The re- 
viewer would have preferred to have seen greater encouragement 
given to the reporting of confidence intervals and/or standard er- 
rors of validity and reliability coefficients. In addition, a suggestion 
to explore possible non-linear relationships between novel tests or 
criteria, particularly when concerned with non-cognitive variables 
would seem to have been in order. 

The excellent “Reliability” section emphasizes the determination 
of components of error variance in tests, a distinct improvement in 
approach. No longer are coefficients classified into types; the use of 
suitable descriptive phrases that convey the meaning of reported 
coefficients is encouraged. 

The brief section on “Administration and Scoring” parallels the 
general high quality of the document. The reviewer would have 
preferred, instead of recommending that the “correction for guess- 
ing” formulas be applied on non-power tests, that experimental 
data supporting the efficacy of the particular method employed be 
provided. To the reviewer's knowledge, the relative value of a 
“correction for guessing” formula in all situations has not been 
definitively established. It does not seem inconceivable that in 
some situations the gambling set would be positively correlated 
with a criterion; hence corrected scores might have less validity 
than uncorrected scores. 

The final section on “Scales and Norms” commendably recom- 
mends the use of standard scores in reporting test results more 
strongly than was done formerly. With respect to the definition 
and meaning of grade placement scales, the reviewer would have 
preferred some encouragement toward increased uniformity. Some 
such direction would probably help reduce the unnecessary lack of 
comparability of grade equivalence from test to test. 

In summary, this document is an outstanding example of high 
quality of content expressed in succinct, non-technical language, 
both of which will greatly contribute to its usability. Every major 
objective of the report appears to have been abundantly achieved; 
in fact, the product might aptly be considered as typifying a stand- 
ard for such standards. 


Kennet D. Hopxins 
Laboratory of Educational Research 
University of Colorado 
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The Clinical Psychologist 


Clinical psychology, to which the testing movement owes much of 
its early origins (and present plight), has an important stake in the 
continuing health and vitality of the movement. Any effort that 
will help the clinical psychologist to select from the many and 
varied test procedures now available and in development is bound 
to have a salutory effect on the currently chaotic scene. The criteria 
developed for reporting test information in the current Standards, 
if accepted by those who write test manuals, should ease the prob- 
lem considerably. 

Clinicians are singled out for mild rebuke on the problem of 
quantification of the results of clinical instruments, particularly the 
projective techniques, The claim that projective techniques cannot 
be submitted to the same standards employed in evaluating other 
psychological tests is examined briefly and, in the main, dismissed. 
Much more on this particular topic can be found in Zubin et al. 
(1965), who argue for the return of projective techniques to the 
psychometric tradition. An additional implied criticism of test val- 
idation of clinical instruments is the collection of data on the basis 
of availability. The number of studies conducted on the basis of 
searching through hospital or clinic files after reading an article on 
a new scoring method for an old technique is beyond calculation. 

Tn addition to these comments, clinical psychologists will find 
many lively topies in this jam-packed pamphlet which will have 
particular relevance for testing in clinical situations. The com- 
mittees, for example, suggest that test manuals, where appropriate, 
report on the results of fakability studies. This might be expanded 
to a suggestion that a “faking” key be developed, put through @ 
try-out, and reported. Concern for acquiescence response set Or 

yes-no” type responses is also expressed, along with suggestions 
for its detection. The problem of “base rates” for diagnostic instru- 
ments appears to have been ignored, although the amount of mis- 
classification or overlapping is listed as an essential item for in- 
clusion in the test manual. How well will a test work in a practical 
situation if, in the validation study the cut-off score makes very 
few errors in distinguishing between fifty normals and fifty schiz- 
ophrenics? While the base rate can be expected to vary from one 
practical situation to another, the reader of a test manual might be 
interested in the answer to this question in selected situations. 

The problem of validity is simplified into three aspects: content, 
criterion-related, and construct. Most of the suggestions, as might 
be expected, are centered iteri “iene 
ine ores 3 a ed on criterion-related validity. Perhaps the 
ME U e e the problem is content validity: the na- 
pling “The Reinga g sampled and the adequacy of the sam- 

ards suggest that “the manual should justify the 
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claim that the test content represents the assumed universe tasks, 
conditions, or processes (p. 12).” How well does a series of draw- 
ings, to which subjects make up stories, represent a universe of 
situations? What is the universe of items from which questions on 
a personality questionnaire should be drawn? The reader can see 
the impossibility of answering these questions at the present time. 
Psycholinguists are becoming concerned with this problem (Cole- 
man, 1964), and a similar concern may take place in psychological 
testing. 

When test manuals of assessment procedures now employed in 
clinical situations are evaluated by the criteria in the Standards, it 
will become apparent that clinical psychology will become the 
chief beneficiary of the new standards. If psychologists of all fields 
will insist on these standards in the test manuals of the future, the 
effect cannot help but be healthy and profound. 
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The Measurement and Evaluation Specialist in a Large Public 
School System 

The authors of Standards for Educational and Psychological 
Tests and Manuals are to be commended for their efficient and 
easy-to-read presentation of so many well conceived recommenda- 
tions which in some cases utilized some rather involved concepts. 
Only occasionally are there sections that discuss concepts which 
exceed the technical competencies of the typical educator. In those 
areas where educators may encounter cognitive difficulties in meas- 
urement concepts the document has presented background informa- 
tion to the reader. In addition, examples and comments are pro- 
vided to explain the standards. The profession should salute the 
cooperative efforts of such a distinguished group of mental meas- 
urement authorities for their extended dedication to complete the 
difficult assignment. 

To the educator especially, criteria for determining certain as- 
pects of the quality of tests and test manuals are strongly needed 
because of the variety of backgrounds and training in educational 
measurement of teachers, principals, and guidance workers and be- 
cause of the many and sundry tests for school use that have man- 
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uals with diverse emphases. The document has developed such cri- 
teria. Although the publication may be used by the most astute 
student of mental measurement, it lends itself well to the school 
teacher, counselor, and administrator. The Standards include ex- 
amples based on subjects ranging from reading readiness tests of 
the kindergarten child to college and vocational placement tests, 

In an era where commercialism has been developed to a high de- 
gree in all aspects of business, it is quite appropriate that the pub- 
lication remind test publishers that manuals, “must avoid using 
high-pressure advertising techniques.” For many of the currently 
used tests to include all the recommendations, although highly de- 
sirable, would pose a rather monumental task on their part. How- 
ever, it is hoped that test publishers will accept the document as & 
standard and construct their tests and manuals in keeping with the 
recommendations in accordance with the standard as conscienti- 
ously as they would adhere to a code of ethics. 

The format of the document is such that it includes for purposes 
of clarification through example strengths (as well as deficiencies) 
of many tests in the field. These models are used frequently as bases 
for many of the standards. 

Although this reviewer feels that the Standards could be ad- 
hered to without undue stress by test publishers, one statement, 
(C5.3), has elements of fantasy to expect test publishers to admit 
that any of their validity samples are, “made up of records ac- 
cumulated haphazardly.” One area untouched by the standard 
which would be of special assistance to elementary school edu- 
cators is that of the use of practice tests for students. Primary 
teachers frequently seek inquiry regarding the use of practice tests 
prior to the administration of pupils’ first tests or types of tests 
new to pupils. 

A concern of the present reviewer is that in section D1.5 from 
the background of D1.4 a reader may become confused that a sta- 
tistical significance of difference between two scores of a profile is 
associated with reliability per se. Students of measurement and 
school staffs who are consumers of tests, all too frequently, become 
confused from their own self-imposed misconceptions of this dif- 


one Nothing in the Standards should enhance this misconcep- 
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changes in means and standard deviations between tests admin- 
istered one year apart, two years apart, and three years apart.” 

The usefulness of the Standards to the school workers can be 
tremendous. The document as a potential to assist educators to 
assess more effectively tests and manuals is indisputable. How- 
ever, the previous Standards often were not utilized or publicized 
to the staffs of a large portion of school districts in the country. As 
these revisions have been made of the 1954 and 1955 Recom- 
mendations, so too, should ways for implementing the present 
standards be reviewed. It is the challenge of present educators to 
develop procedures to publicize and to implement the standards. 
Educators could publish a supplementary document which would 
provide additional recommendations for school use. Three examples 
of materials that could be included in such a document are recom- 
mendations for establishing elements of a good testing program, 
ways to provide in-service training of test interpretation to teach- 
ers, and characteristics of a good board-of-education report of a 
district’s test data. 

With regard to the myriad of schools and school districts, those 
agencies which have responsibilities to serve schools have a prob- 
lem of how to disseminate and implement the standards. Whether 
the Standards become or are used as a tool to improve the use of 
tests so that schools receive the benefits of the intent of the Stand- 
ards is a function of those staff members in schools and district 
office personnel who are responsible for testing programs. It is rec- 
ommended that educators establish local and regional meetings to 
discuss implications of the Standards at their respective level. 

Local, regional, and state educational organizations are urged to 
coordinate the assessment of tests and manuals according to the 
Standards. 

Jerry G. GARLOCK 
County Schools of Los Angeles 


The Textbook Writer in M easurement and Evaluation 


One of the truly helpful documents for test authors, test pub- 
lishers, and test users was the supplement to the Psychological 
Bulletin entitled Technical Recommendations for Psychological 
Tests and Diagnostic Techniques published in 1954. Beyond a doubt, 
this publication represents a kind of benchmark in a series of steps 
leading to the establishment of widely accepted standards for psy- 
chological tests and inventories. Today, it is widely mentioned and 
quoted by the members of the audience to which it was addressed. 
Moreover, it tended to standardize terminology in testing as it ap- 
pears in journals, books, and test manuals. Finally, it generated a 
number of useful manuscripts, including a number of journal ar- 
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ticles concerning construct validity and a companion publicati 
entitled Technical Recommendations for Achievement Tests which 
appeared in 1955. 

Over ten years have passed since the Technical Recommendas 
tions were published. Now, a revision is available. It, too, is the 
product of extensive committee work. The product of the efforts of 
the committee is truly another contribution of significance. The 
revision concerns both psychological tests and inventories as well 
as educational tests, and thereby eliminates the need for the two 
existing publications. 

The audience for the revision is described as one which is fairly 
sophisticated in testing matters. The committee intended that th e 
material included be meaningful to those who have a level of for- 
mal training between the master’s degree and doctorate in educa 
tion or psychology at a superior university. Obviously, the class 
Toom teacher who may have had little or no formal education in 
testing is excluded. This is regrettable, since this very large group 


the standards themselves, As in the case of the 1954 publication, 
six major subdivisions are included; namely, Dissemination of In- 
formation, Interpretation, Validity, Reliability, Administration 
and Scoring, and Scales and Norms. The most significant of the six 
are the two dealing with validity and reliability. The remaining 
are shorter and in some ways less important. 4 
The subdivision concerning dissemination of information and 
the subdivision concerning interpretation resemble greatly those of 
the 1954 Publication, Two observations are in order, however. First ~ 
of all, the point is made in several instances that test manuals 
should indicate clearly that which 
yielded, Warning should be 
be thought of as being me: 
1s not. Realistic cautions 
invaluable. It wi 


more years old and no revision can be obtained.” Although one can ~ 
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quarrel with the limitations of 15 years, one cannot quarrel with 
the intent of this standard. Revisions of tests and test manuals 
often appear too infrequently. 

Even the casual reader of the Standards will quickly discover 
that the discussion of validity is changed in one major way. 
Whereas the 1954 publication cited four kinds of validity, the re- 
vised publication mentions three. Content and construct validity 
remain virtually unchanged. The original concurrent and predictive 
validity had been combined into one category known as criterion- 
related validity. This makes sense, since the only important dif- 
ference between concurrent and predictive validity is the time ele- 
ment. Emphasis on criterion-related validity is consistent with 
certain recent publications concerning criterion-related research. 
Such a categorization of research and categorization of validity re- 
duces the need for detailed description of both. 

A point of major emphasis in the validity standards as well as the 
reliability standards is sampling. Certainly, there has been a tend- 
ency by test authors and test publishers to underestimate the 
importance of this feature of their tests. Samples of overt behavior, 
samples of subject-matter areas, and samples of subjects are 
sometimes mentioned briefly with little attempt to defend the 
strength of the sampling process used. Indeed, sometimes the popu- 
lation which the sample is thought to represent is barely described. 
The significance of this point is presented well by several standards 
concerning criterion-related validity. Important criteria to be ap- 
plied in evaluating the worth of criterion-related validity deter- 
minations are listed. 

As in the case of the validity subdivision, a noteworthy change 
has taken place in the reliability subdivision. In addition to mod- 
ernizing parts of the discussion, the committee decided to abandon 
some of the terminology used in the 1954 report. Rather than 
speaking of coefficients of equivalence, coefficients of stability, and 
the like, it prefers a more complete statement about the reliability 
coefficient. This statement would briefly describe the nature of the 
reliability determination, and would not necessarily give it a title 
such as those previously used. 

The section concerning administration and scoring is one of the 
shorter sections and is essentially complete insofar as the audience 
of the manuscript is concerned. This brevity, however, should not 
be interpreted to mean that the standards listed are necessarily of 
Jess importance than others. Some of the practices found in schools 
with regard to test administration and scoring are deplorable. State- 
ments in test manuals are often clear and emphatic. Somehow, 
however, the message is partially or totally lost to some teachers. 
One of the frustrations of the testing profession is its inability to 
communicate with the test users—for example, teachers—with re- 
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gard to the importance of standardizing administration and scoring 
practices. 

In short, it must be said that the 1966 revision is a fine improve- 
ment over the 1954 publication. One of the strengths of the re- 
vision yet unmentioned is the set of parenthetical comments which 
provide examples of violations of a particular standard or success- 
ful adherence to it. Although the 1954 publication used this tech- 
nique as well, the parenthetical comments are more numerous and 
more pertinent in the 1966 revision, 

The addition of the index is excellent, Although it is not an es- 
sential part of the publication, it certainly is a helpful part. 
Another useful addition might have been a selected bibliography 
concerning standards for tests and manuals. It is obvious that 
particularly good readings exist in the areas such as validity and 
reliability, Perhaps in another decade still another committee will 
oe the current edition and see fit to make this improvement 
as well. 


J. STANLEY AHMANN 
Colorado State University 


Personnel Psychologist in a Large Industry 
Possibly never in the history of psychology has so much cogent 
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augmented in the new publication; the cautions mentioned relative 
to construct validity should be carefully considered by every test 
publisher and user. 

Some persons actively engaged in employment psychology may 
not feel that Standards for Educational and Psychological Tests 
and Manuals is responsive enough to some of the social and tech- 
nical issues facing the personnel psychologist today. It is to be 
noted, however, that it is specifically stated that, “. . . primary 
responsibility for improvement of testing rests on the shoulders of 
test users.” A further consideration is that some of the presently 
most pressing problems in employment psychology are likely to be 
of a temporary nature and will probably not be so important when 
educational opportunity is more nearly equalized throughout all 
socioeconomic levels. 

There are, however, in the document several points which every 
personnel psychologist should consider. Among these are strictures 
against implying that a test measures an “innate” ability and using 
the labels, “culture-free” and “culture-fair.” The cautions expressed 
relative to the use of moderator variables should be exercised in 
employment research. 

From an editorial viewpoint, the publication is excellent. A 
rather complete index will aid in making Standards for Educa- 
tional and Psychological Tests a more useful reference than its 
predecessor. 


Mary L. TENOPYR 
North American Aviation, Inc. 


Language Testing: The Construction and Use of Foreign Language 
Tests by Robert Lado. New York: McGraw-Hill Book Co., 
1964. Pp. xxiii + 389. 

In an age of such unprecedented and universal change as ours it 
may seem presumptuous and pointless to single out change in any 
one educational field. Still, the upsurge of interest in language 
teaching and the curricular changes resulting from it present one 
aspect that is probably unique. While few if any American educa- 
tors ever seriously advocated dropping science, or mathematics, or 
social studies, from the secondary school curriculum, a sizable part 
of the educational community considered the teaching of foreign 
languages, classical or modern, largely a waste of time. The events 
of the last thirty years have taught us that the world outside our 
borders cannot safely be ignored and that, in order to coexist with 
other peoples, it is necessary to understand and speak their lan- 
guages. When we took a good look at the type of language teaching 
going on in our schools, however, we were dismayed to discover 
that it was indeed a waste of time. The student who left college— 
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not to mention high school—with a functional knowledge of a lan- 
guage he had studied was the exception rather than the rule. 

The effectiveness of recent curriculum changes is due to the fact 
that they are not superficial reforms of traditional methods but are 
rooted in a basic new understanding of the nature of language, 
Modern linguistic science has struck down the artificial categories 
into which language had been pressed many centuries ago and cre- 
ated new concepts and categories to fit language as it is. The new 
methods arising from this new understanding are increasingly ap- 
plied in schools throughout the country. It is the purpose of Dr. 
Lado’s book to provide teachers with adequate techniques for eval- 
uating student learning in courses utilizing these new methods, A 
well-known linguist, Dean of the Institute of Languages and Lin- 
guistics at Georgetown University, and the author of Language 
Teaching: A Scientific Approach, Dr. Lado is eminently fitted for 
this task. His book is very well organized. Every aspect of the field 
is covered fully and concisely from the point of view of linguistics, 
teaching methods, and measurement. A detailed content outline 
covering 15 pages will effectively guide the teacher seeking infor- 
mation on any specific problem. Part I provides an orientation to the 
linguistic thinking as well as to the premises about language testing 
on which the book is based. The teacher unfamiliar with linguisties 
might do well to peruse a few of the reference works listed before 
attempting to put the book to practical use. Parts IT and ITI discuss 
in detail the variables to be tested as well as suitable techniques 
for testing them. Part IV deals with the special problems of test- 
ing cross-cultural understanding which, while transcending lan- 
guage, is yet intimately related to it. The technical aspects of 
measurement are explained in detail in Part V. 

__ The author Points out that some of the techniques generally used 
in language testing, such as translation, dictation, or essay writ- 
ing, have little relevance to the skills they claim to measure, and 
that the same is true for objective tests constructed without a syste- 
matic understanding of the variables involved in language learning. 
: CoN analysis has made available the means for accurately 
identifying the elements of language as such as well as those ele- 
monte integrated in the recognition and production skills. Teach- 
Nie Ai Hd isolate the problems of language learning and test 
Gebnlene eTa far beyond that achieved by the haphazard 
e a Liy tradition. In their place, teachers will find 
erkek a of techniques to fit different needs, and an 
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pitfalls of item writing; some others, however, have been over- 
looked. 

The book has other weaknesses. It is an unchanged reprint of the 
London edition of 1961 and, judging from the date of the author’s 
preface (1959), it is even older than that. In a field that has ad- 
vanced rapidly in the past few years this is a serious shortcoming. 
The two major instruments of standardized foreign language test- 
ing, the Modern Language Association’s Proficiency and Coopera- 
tive tests, which incorporate many of the ideas presented by Dr. 
Lado and are by now widely used, are not mentioned, Nor are the 
many tests included in recent textbooks. Surely a few short para- 
graphs on developments as crucial as these could, and should, have 
been added. Other references are similarly out of date. The value 
of the book would have been greatly enhanced by incorporating in 
it the results of the nationwide experience with standardized, 
normed test batteries, for instance in the area of constructing 
and scoring Speaking tests. 

Much of the experience on which the book is based was gained 
through the teaching of English as a foreign language. Teachers of 
foreign languages should not be deterred by this. It may make 
them aware of problems in their own teaching which they had not 
been conscious of before. The few instances in which it has led to 
inappropriate conclusions can be disregarded. However, teachers 
may be seriously discouraged by the insistent recommendation to 
validate tests with native speakers. This is unrealistic except 
where a language is being taught in its native environment. It 
also has a number of pitfalls and limitations that are only in- 
completely pointed out, while most of its benefits can be procured 
by having a test reviewed by one or two qualified language teach- 
ers. Also, teachers familiar with current theories of modern lan- 
guage learning will be taken aback by the frequent use of the na- 
tive language as a stimulus. 

The book has no index. Although it is largely replaced by the 
content outline, the teacher seeking information on some topic 
treated earlier would profit from page references. This drawback of 
the book’s praiseworthy conciseness could easily have been avoided. 

Not all of Dr. Lado’s suggestions can be accepted without quali- 
fications. Factors extraneous to the problem tested are pointed out 
in some techniques but not in others, e.g. lexical recognition in 
testing production by sentence completion or the omitted letters 
technique. While the professional test writer might be able to sug- 
gest more appropriate techniques in some cases, or to point out 
some flaws in item writing that might have been avoided, even he 
can profit greatly from the theoretical and practical ideas pre- 
sented in this book. 

The author makes it clear that this is a teacher’s book, and it 
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will indeed be invaluable to the language teacher. For the language 
supervisor, it) will be of great help in the construction and use of 
more formal tests for which the classroom teacher lacks both time 
and facilities. It should also be recommended reading for admin- 
istrators and guidance experts. The essential difference between 
language as a vehicle for the communication of ideas and other 
subjects that consist of ideas as such, the fundamental differences 
between testing of the native or of a foreign tongue and, above all, 
the realization of the many and complex variables that enter into 
language learning, should come as an eye opener to those who have 
thought of a foreign language as just another subject, and of & 
foreign language test as just another achievement test. 


Suzanne STAHL A 
Educational Testing Service 


Acquiescence and Social Desirability Response Styles, Item Char- 
acteristics, and Conformity by Lawrence J. Stricker. Psycho- 
logical Reports Monograph Supplement 2-V12, Missoula, Mon- 
tana: Southern Universities Press, 1963, Pp. 22. $1.00. 


The research reported attempts to integrate conceptualizations 
from two quite different origins: experimental studies of conformity 
and Personality research on the response styles of acquiescence 
and social desirability. The first of these concepts involves a sub- 
ject’s Yielding to opinions expressed by others in the experimental 
situation and the second involves the subject’s either generally 
eereeing with paper-and-pencil test items or agreeing with those 
items that reflect socially acceptable norms, Stricker argues that 
pre-established norms which a subject carries around with him 
may affect his responses on Personality and attitude tests in the 
ways that the norms established by experimental “stooges” affect 
responses in conformity experiments. If so, then there also may be 
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all, 27 male and 66 female graduate and undergraduate psychology 
students were used. Forty-six subjects were dropped because of 
some “awareness” of the experiment’s purpose (a procedure that 
might have produced some sort of harmful subject selection). At 
the same time, subjects were answering the experimental items, 
which consisted of 73 derived from the Social Adjustment subtest 
of the Minnesota Multiphase Personality Inventory (MMPI) and 
65 derived from Ferguson’s Humanitarianism scale. There were 
16 variations in the items based on four variables: clearness, ex- 
tremeness, social desirability, and positive vs. negative phrasing. 
Independent manipulation of the last two variables allowed 
Stricker to untangle acquiescent responses from social conformity 
responses. 

In the main, Stricker found powerful effects of item variation 
and virtually no effect of personality-response style. More acquies- 
cent responses and fewer socially desirable responses were made 
to unclear than to clear attitude items. This outcome would be 
expected, since unclear items do not allow the subject to identify 
social norms to which he might conform. With the personality 
items, fewer socially conforming responses were made to unclear 
items than to clear items; fewer acquiescent responses were also 
made to these items. This result fails to agree with the hypothesis. 
More acquiescent responses were made to moderate personality and 
attitude items than to extreme ones, again presumably because it is 
more difficult to identify social norms with moderate items. How- 
ever, although more socially conforming responses were made to 
extreme personality items than to moderate ones, no reliable dif- 
ferences were found for extreme and moderate attitude items. This 
outcome, again, is contrary to hypothesis. 

Stricker had hypothesized that there should be a strong positive 
correlation between the two kinds of responses that show most con- 
formity: acquiescent responses to moderate items and socially con- 
forming responses to extreme items. The obtained correlation of 
—21 was significant at the .05 level. Taken together with absence 
of reliable differences as a function of either of the personality 
response-styles, this result casts doubt on the conception of a gen- 
eral personality dimension of conforming. Perhaps, after all, con- 
forming like suggestability is more a function of the conditions of 
decision than of the individual’s long-term characteristics. Stricker 
cites many previous studies, the findings on which show that these 
two response-styles do not relate to amount of conformity in ex- 
perimental manipulations. Thus the situational point of view is 
supported. 

‘Although Stricker discusses theoretical possibilities regarding 
the inconsistencies between his hypotheses and data, it seems 
likely that experimental artifacts may have produced them. For one 
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thing, his method of obtaining moderate and extreme items and his 
criteria for assessing them are not very valid. Many times items 
were made “more extreme” by adding extreme frequency words like 
“always” and imperatives like “must.” Such a procedure does not 
really produce a more extreme social attitude; it is the same old 
attitude held more extremely. The items were then classified by 11 
graduate students in psychology for extremeness, clearness, and so- 
cial desirability—and again for social desirability by 40 under- 
graduates. Samples of such size are not really large enough for one 
to formulate important judgments. Indeed, Stricker found: (a) 
that the two samples disagreed radically on the social desirability 
of the attitude items, (b) that the clear and unclear items did not 
differ—either on the Flesch or on the Dale-Chall readability in- 
dex, and (c) that (most damaging of all) the socially desirable 
items were rated more “moderate” than the socially undesirable 
items and vice versa. 

Therefore, one might expect: (1) The measurement of a number 
of socially desirable responses was not completely reliable for the 
experimental subjects. (2) The effect of item clearness was rather 
vitiated in the experiment. (3) Since moderateness per se of the 
items in this study was more socially desirable than was an ex- 
treme position, the subject should be more likely to agree than to 
disagree with any moderate item (artifactually). Moreover, there 
should be an artificial depression in the number of socially con- 
forming responses to moderate items (since “undesirable, mod- 


erate” items were not really so undesirable as might be antici- 
pated). 


Apert SILVERSTEIN 
University of Rhode Island 


Psychodiagnostic Testing: An Empirical Approach by Molly Har- 
cls ener Illinois: Charles C. Thomas, 1965. Pp. xxi 


Dr. Harrower had an intriguing idea, one which must have oc- 
curred to every psychologist engaged in the practice of individual 
psychological testing: “What has happened to all the patients I 
mE ra down P the years?” While this question may 

sed a momen stir for most chologists, Dr. Har- 
rower set out to find the answer. The eies this elim volume, a 
follow-up study of tested patients referred to Dr. Harrower by 
psychotherapists of various persuasions. 

The purpose is certainly heroic. By analyzing the test records 
after at least four years has elapsed since testing was accomplished, 
the author is attempting to determine the test predictors of the 
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outcome of psychotherapy. Dr. Harrower was able to put together 
a sample of 622 subjects, an awesome size for a study with this 
purpose. From the records the author devised a rating scale, the 
details of which make replication almost impossible. 

The test battery consisted of the Wechsler-Bellevue, Rorschach, 
Draw-A-Person, Sentence Completion, Thematic Apperception 
Test, The Most Unpleasant Concept, and the Szondi. Just how 
these varied procedures, some with minimum validity and others 
considered to be monuments to the naivité of clinical psychologists 
in the area of psychometrics, play a part in what comes later is 
hardly mentioned. Based in some way, but not detailed, on test per- 
formance, the subjects were divided into degrees of “mental health 
potential.” Dr. Harrower attempts to divide mental health poten- 
tial into categories similar to those used in categorizing subjects 
tested with intelligence tests (ie., “average,” “dull,” “bright nor- 
mal,” etc.). This effort to overcome the problem of sorting the 
results of projective test into psychiatric categories, however, 
overlooks one important problem. Wechsler’s scheme for divisions 
of intellectual status, the basis for Harrower’s hope, is firmly tied to 
the properties of the normal curve. Does this projective typology 
share the same property? 

Although no cross-validation of the results (subjects with posi- 
tive mental health are rated as more successful in psychotherapy 
by the therapists) is presented, the author did check the concept in 
a different problem, the selection of ministers. Unfortunately, no 
correlations or significant levels are reported, although the “eye- 
ball” evidence appears to be in line with her notion that mental 
health potential is an important construct. 

Of interest to clinical psychologists is the material in this book 
devoted to the results of psychotherapy. In general, the trend is 
marked for orthodox psychoanalysts to rate themselves as most 
successful, and with departures from orthodoxy, the claims for 
therapeutic success are reduced. This discussion of therapeutic suc- 
cess is difficult to evaluate. Out of Eysenek’s disparaging survey of 
the results of psychotherapy has come an appreciation of the method- 
ological difficulties in evaluating therapeutic success. Unfortu- 
nately for the reviewer, Lawrence S. Kubie’s foreword disarmingly 
admits to many defects in Harrower’s study. A few others might 
also be thrown in: lack of control of the experience level of the 
therapists, a laissez faire approach to the problem of criteria of 
therapeutic effectiveness, and an avoidance of the issue of control 
groups. To be sure, it would be difficult to devise a meaningful con- 
trol group (such as “no treatment”) for a study based on patients 
referred by psychotherapists. But surely the author’s files contain 
many subjects not referred for pre-psychotherapy evaluation and 
who received a similar test battery. Some method for follow-up 
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could be developed, with both groups being rated by someone else 
besides the therapist for the experimental group. 

It is the reviewer’s opinion that this overpriced book might 
better have been presented in the form of a journal article. With 
none of the page restrictions faced by a journal contributor, the 
insufficient description of the procedures, along with sketchy case- 
history material on validation, does not justify the hard cover. 


PHILIP HIMELSTEIN 
Texas Western College of the 
University of Texas 


Basic Statistical Methods (Second Edition) by N. M. Downie and 
ae Heath. New York: Harper & Row, 1965. Pp. xv -++ 325. 


Basic Statistical Methods is intended as an introductory text for 
social science students who generally have a limited mathematical 
orientation. This objective is successfully attained, since the treat- 
ment is clearly written and quite comprehensible. The chapters 
are short and devoid of verbiage; a real boon. As the print contrasts 
well with the dull-surfaced paper, eye strain is minimized. Ap- 
propriate, relevant, and easy-to-read tables are furnished in the 
Appendix. Its format is such that a fairly good balance of at- 
tention is assigned to describing assumptions, presenting sufficient 
but not tedious derivations for adequate comprehension of formu- 
lae and relationships, citing common formulas with illustrative 
computations, furnishing usual probability interpretations, and in- 
rene pome practical applications. i 

avorable reactions to the text were obtained from an education 
arid ae teachers and/or counselors. Their major comments 
Pg E pleasure with the light treatment of derivation, with 
ae Patra easy-to-follow meaningful examples, and with the 
ility. Some students thought that smaller numbers could be 


used in certain assigned i 

: problems so that more study tim Id b 

applied to understandin; icati a toed 
ating on AB oat g and application rather than to concen- 


ations. Moreover, they expressed a desire to 
pote E vs cS labelling certain problems, conditions, 
a e situations and not to function only as 
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Although this reviewer is well satisfied with the text and will be 
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likely to continue using it with future classes, some suggestions and 
constructive comments are offered as a result of his experiences 
with the text this past semester. In general, more practical appli- 
cations from the social sciences should be included. Any designa- 
tion of a group or condition should be implemented with descrip- 
tive, real life examples, not merely by use of the letters A, B, and 
C or of 1, 2, and 3 type labels. 

The eighteen chapters comprise the main topics usually found in 
most introductory statistical texts except for the chapter on non- 
parametric statistics, (A brief introduction, which lists the reasons 
for studying statistics, is followed by a short history of contributors 
to statistics. A section on the usefulness of statistics rounds out 
Chapter 1.) 

In Chapter 2 the typical arithmetic fundamentals, types of meas- 
urements, measurement scales, and statistical symbols are treated. 
Only one topic caused minor concern: the term significant digits is 
never defined although practical, obvious examples for depicting 
the rules are shown; thus the definition may be inferred. In terms of 
reducing the number of statistical symbols to differentiate sample 
or population values, one might simply use a tilde over the symbol 
for signifying parameter values and thereby reduce the total num- 
ber of symbols by one half. 

In Chapter 5 on variability, a label to identify the first row of 
numbers in Figures 5.1 and 5.2 as raw scores would help. In Figure 
5.3 the order of the scores from high to low could be inverted ad- 
vantageously to low to high to coincide with the graphic descrip- 
tion on “error of coarse grouping” as is ordinarily the case. Rather 
than imply, the authors should explicitly state that error of coarse 
grouping tends to enlarge the estimates of sigma and that Shep- 
pard’s correction provides the corrected value. An example would 
be helpful. 

The section on “Correlation” (Chapter 7) could have a segment 
specifically devoted to an interpretation of the meaning of this 
term. The Pearson r is thoroughly described in Chapter 7, but its 
interpretation and significance are not spelled out until Chapter 18. 
With reference to the table of significant values of r, discussion that 
the probability of an observed correlation as large as or larger 
than the one found being due to chance, would be suitable imple- 
mentation. 

Chapter 8, “Linear Regression,” erroneously calls bys a Beta co- 
efficient (p. 98). Two additional formulas are suggested for inclu- 
sion to complete the methods of computing the b-coefficients be- 
oe of their common usage: bys = Tys 8/82 and its counterpart 
for bay. 

The formula, ss = \/Npq, from Chapter 10, “Sampling,” should 
be related to the section, “Use of the Formulas of the Binomial 
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Distribution,” of Chapter 9, Probability and the Binomial Distribu- 
tion. Practical examples dealing with smoking preferences, coca cola 
tasting, and number of items right in objective tests are suitably 
depicted with the binomial formula (e = 1/npq) in Chapter 9, but 
its counterpart is glossed over in Chapter 10. 

Particularly well-written, Chapter 11, “Testing Hypotheses, The 
Differences between Means,” might be improved in one minor way. 
It could be specified that t or z tests apply (depending on sample 
size) to tests of significance of differences between means for cor- 
related data (computed directly from differences without comput- 
ing a correlation coefficient) in the same section in which the means 
and correlation coefficient are given rather than two pages later, 
A table with two columns listing “Conditions” in one column 
(small or large samples, correlated or uncorrelated data, etc.) plus 
the “Formulas” in a second column could succinctly summarize 
the entire chapter. In fact, such a table was constructed as a series 
of matching items in part of one of the reviewer’s course examina- 
tion. In the latter table, however, actual sample sizes, methods of 
sample selection, and other information were substituted for the 
“conditions” Suggested above in order to promote understanding, 
not mere recapitulation of facts, 

Perhaps a better appreciation of the Lawshe-Baker Nomograph 
for testing the significance of differences between percentages or 


An exceptionally clear presentation is made in the “analysis of 
Variance” portion of Chapter 15. Unless a 


volve sample sizes rarely exceeding 30 or 35 cases, In addition, the 


c € ation of these two statistics to es- 
tablish evidences of relationships and/or reliabilities of behavior 


rankings are especially pertinent. Finall: i 
I 1 ly, by using the same set: 
of numbers, where possible the relative ‘advantages of the es 
statistics could be more lucidly demonstrated. 
Most of the Suggestions noted are Proposed with the main idea 
that many social science students enroll in only one statistics 
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course. Their primary interests are with applications involving 
data about people, not with abstract research concepts or numbers. 
They have generally avoided courses emphasizing research meth- 
odology and statistical thinking in the past. If statistics is featured 
in less foreboding ways, probably some of their resistance would 
dissipate and more research application might result. Many stu- 
dents who have excellent ideas for investigation are reluctant to 
pursue them because of the numerical “bugaboo.” It is probably 
more efficacious to make the subject of statistics as palatable as 
possible and still to meet many of their practical needs. Another 
important reason for a nonthreatening presentation is that stu- 
dents can become improved consumers of published works—con- 
sumers who are more aware of cogent questions to be raised when 
evaluating published research. This point is stressed, since many of 
these same students eventually become administrators who effect 
decisions of varying and costly magnitude even though they lack 
even minimal statistical knowledge. In its present form this text 
should make definite contributions toward realization of these de- 
sired teaching objectives. 

Rosert R. MORMAN 

California State College 

at Los Angeles 


Introduction to the Theory of Statistics by Alexander M. Mood 
and Franklin A. Graybill. New York: McGraw-Hill Book Com- 
pany, 1963. Pp. xv + 443. $8.95. 


This book is the second edition of a very popular introductory 
mathematical statistics book. The first edition developed from the 
lecture notes prepared for a course given to seniors and first year 
graduate students by the first author while he was at Iowa State 
University. Neither edition assumes any statistical background, 
but each does assume one year of undergraduate calculus. How- 
ever, the merits of the books, especially this newer edition, will be 
better appreciated by those readers who have had a previous course 
in introductory statistics. Matrix algebra is used quite heavily in 
discussing the multivariate distribution, multiple regression, and 
experimental design; and although a section is devoted to the ele- 
mentary properties of matrices and determinants, the reader would 
probably benefit by having a little prior knowledge of matrices and 
determinants. 

The text would probably be of little interest to the average 
behavioral or social science graduate student because of the rela- 
tively rigorous mathematical treatment of the subject matter. On 
the other hand, the graduate student who is anticipating an ex- 
perimentally oriented research career, or the student and prac- 
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titioner of psychological statistics would certainly do well to read 
this volume. This introductory book fills many gaping holes that 
appear in the average statistical texts oriented toward the be- 
havioral and social sciences. Many of the statistical concepts 
found in this book should also be found in the psychological sta- 
tistics texts, but disappointingly this is rarely the case. 

The volume is comprised of sixteen chapters. Each chapter con- 
tains numerous illustrations, ample problems, and a welcome up- 
to-date bibliography. In the review of this book an attempt will be 
made to emphasize those topics that rarely appear or are given 
inadequate coverage in most of the psychological statistics texts, but 
are given lucid and comprehensive treatment in Mood and Graybill s 
text. The chapter on probability, which is fairly comprehensive, cov- 
ers such topics as set theory, sample spaces, marginal probabilities, 
conditional probabilities, and other fundamental concepts of prob- 
ability. Since probability is the cornerstone of statistics, a fairly 
comprehensive discussion of this subject is in order. There is a 
chapter for both discrete and continuous random distributions. 
The multivariate distribution is discussed for both cases as well as 
for marginal distributions. The concept of the moment generating 
function which is an important tool for defining distributions is 
discussed and applied continuously throughout the book. 

The chapter on point estimation is one of the most unique and 
informative chapters in the text. In this chapter, the authors dis- 
cuss such fundamentally important concepts as sufficient statistics, 
and minimum-variance unbiased estimates. The sections on likeli- 
hood estimation and Bayesian estimation are extremely well writ- 
ten. One whole chapter is devoted to the multivariate normal dis- 
tribution with a subsection on quadratic forms. This chapter is 
discussed realistically in terms of matrices and determinants, as 
it should be, With the increasing emphasis on multivariate tech- 
niques, this treatment becomes a valuable part of the book. The 
chapter on sampling distributions not only gives a clear picture of 
how the distributions of various estimators are derived, but also 


shows the development of some of the test distributions such as 


Chi Square and F. The chapter on hypothesis testing is a major 
contribution in that hypoth 


esis testing is discussed in the frame- 
work of modern decision theory rather than in the older classical 
tradition. Likelihood ratio and goodness of fit tests are also dis- 
cussed in this chapter. 


It is refreshing to see multiple regression and experimental de- 


sign developed in the framework of matrix algebra, However, it is 


rather disappointing that experimental design was given such short 
coverage, 


In summary, the Teviewer Tecommends the book to the serious 
student of psychological statistics, The text gives a good basic 
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theoretical background of the development of statistics that will 
lead to more rigorous applications of psychological statistics as 
well as a deeper appreciation for the discipline of statistics. The 
breadth, depth of coverage, and currency of the book also add to 
its attractiveness. 


Grorcs H. DuNTEMAN 
Regional Rehabilitation Research Institute 
University of Florida 


Linear Algebra by George Hadley. Reading, Massachusetts: Addi- 
son-Wesley Publishing Company, 1961. Pp. ix + 290. $7.95. 


It was Hadley’s intention of fullfilling the students’ needs for a 
knowledge of linear algebra in a number of fields such as engineer- 
ing and operations research as well as the social and behavioral 
sciences. Although the book is aimed at students with a limited 
mathematical background, the average graduate student in the 
social and behavioral sciences has probably not obtained this “lim- 
ited” mathematical background. With the increased emphasis on 
multivariate procedures such as factor analysis, discriminant analy- 
sis, and multivariate analysis of variance, some knowledge of 
matrix algebra, the theory of determinants, and n-dimensional 
geometry is becoming more essential for research workers and psy- 
chological statisticians. This book does an excellent job of pre- 
senting some of these basic concepts so essential for an understand- 
ing of multivariate analyses. Although other books such as Thur- 
stone’s (1947) Multiple Factor Analysis and Harmon’s (1960) 
Modern Factor Analysis were written primarily for a psychological 
audience and contain introductory chapters on n-dimensional ge- 
ometry and the theory of matrices and determinants, their coverage 
of this subject matter is neither comprehensive nor rigorous enough 
for the purposes of most students of multivariate analysis. On the 
other hand, treatments of this subject matter by Anderson (1958) 
and Rao (1965) seem to be at a too rigorous and abstract level for 
most researchers. A skillful writer, Hadley presents his topics lu- 
cidly at an intermediate level. There are problems and suggested 
readings at the end of each chapter. 

The book is comprised of seven chapters. Chapter 1, the introduc- 
tion, illustrates the use of linear algebra in the building of linear 
models, Chapter 2 is concerned with vectors and covers many con- 
cepts fundamental to factor analysis and profile analysis such as 
the magnitude and direction of vectors in n-space, the Euclidean 
coordinate system, the concept of distance in n-space, and the 
linear dependence of vectors. 

Chapter 3 covers matrices and determinants. In this chapter 
most of the basic matrices and matrix operations are discussed and 
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illustrated. The section of the chapter concerned with determinants 
is especially informative. Two ways to evaluate determinants are 
discussed while the author is focusing upon the theory of determi- 
nants and at the same time providing numerical illustrations. Ma- 
trix inversion also receives comprehensive treatment. Many of the 
matrix operations described in this chapter are used in factor 
analysis. 

Linear transformations and the rank of matrices are discussed in 
Chapter 4. Chapter 5 is involved with the theory and solutions of 
simultaneous linear equations. This chapter is probably of less in- 
terest and relevance than any of the other chapters. Chapter 6 is 
concerned with convex sets and n-dimensional geometry and is 
probably the most valuable and unique chapter, since the reviewer 
knows of no other books that satisfactorily discuss n-dimensional 
geometry at a relatively concrete and elementary level. The dis- 
cussion of set theory will be of special interest to researchers con- 
cerned with decision theory and discriminant analysis classification 
procedures, since essential concepts such as boundaries between 
regions and hyperplanes are discussed. 

The last chapter, Chapter 7, covers characteristic value prob- 
lems and quadratic forms. The basic determinantal equation that 
underlies principal component analysis is fully discussed. The nu- 
merical illustration of determining latent roots and vectors is es- 
pecially illuminating. The discussion of quadratic forms should be 
helpful to the researcher concerned with discriminant analysis and 
multivariate frequency functions. 

In summary, Hadley’s book is an extremely well written and 
comprehensive reference book on the basic formulations of ma- 
trices, determinants, and n-dimensional geometry that should prove 
especially useful for students and practitioners of multivariate 
analysis, The book in many respects seems almost tailor-made for 
the psychological statistician in that so much useful and diverse in- 


formation is contained between the covers of one 290 page book. 


REFERENCES 
Rao of. spay tora Inferences ai . = ‘ 
Theses Le Analysi: me 
Chicago Press, 1947. Pere bae: Taed 


Grorce H. DUNTEMAN 


Regional Rehabilitati : 
University of Florida Research Institute 


BOOK REVIEWS 781 


Applications of Electronic Data Processing Methods in Education 
by John I. Goodlad, John G. Caffrey, John F. O'Toole, Jr., and 
Louise L. Tyler, assisted by Fred L. Converse and Arthur N. 
Thayer. A study report supported by the Cooperative Research 
Program of the United States Office of Education, Department 
of Health, Education, and Welfare. Project No. F-026, (Los 
Angeles: Dept. of Education, University of California at Los 
Angeles, 1965). Pp. 105. 


The educator who is interested in the application of electronic 
data processing (EDP) procedures in an educational setting has to 
begin his quest for knowledge somewhere. The monograph Applica- 
tions of Electronic Data Processing Methods in Education by 
Goodlad and his associates represents one such starting point. This 
monograph is a report of a conference of EDP specialists and an 
attempt to describe the state of EDP as it existed circa 1964. 

The monograph is one of very few publications with which this 
writer is acquainted that presupposes almost no knowledge whatso- 
ever about EDP and requires no mathematical background in order 
to follow. As such, it should be welcomed by many educator-laymen 
(as far as the computer is concerned) whose ability to read techni- 
cal writing has gone into a positively accelerated decline since their 
own school days. The monograph begins with a general discussion 
of education and computer technology. An attempt is made to relate 
computer technology to the changing American school and to iden- 
tify possible areas of application of EDP procedures to educational 
problems. 

The second section describes some illustrative applications of 
EDP and information processing systems in education. The appli- 
cations range from installations which perform a variety of finan- 
cial and social bookkeeping operations to experimental projects for 
computer based instruction and programs for the simulation of an 
entire school system. Needless to say, the latter applications are 
still in experimental stages and are not likely to be operational for 
some time. 

The third section of the report presents the results of a confer- 
ence of EDP specialists and educators with an interest in EDP 
matters. The conference produced a series of recommendations for 
the use of EDP in an educational setting and suggested criteria for 
the funding of projects to explore additional applications of EDP in 
education. The tone of the recommendations is that our present 
state of knowledge is quite limited and that a number of studies 
should be undertaken to explore the ways in which EDP methods 
can be used in education. The criteria for the funding of projects 
offer precious little in the way of advice to the grant-seeker or, for 
that matter, the grant-giver, e.g., “Will the project advance educa- 
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tional goals by removing a roadblock or otherwise resolving a 
recognized problem?” 

The fourth section sets forth a number of conclusions and recom- 
mendations of the authors concerning EDP. The authors seem to 
have discovered that the computer is a useful tool and could be of 
significant benefit to education, although they are not quite sure 
how this is to come about. They are obviously highly impressed 
with the capabilities of the computer but do realize that technology 
is no substitute for ideas, 

Perhaps the most interesting parts of the monograph are con- 
tained in two appendices. One describes the state of EDP in educa- 
tion, while the other presents thumbnail sketches of twenty-seven 
state, regional, and local EDP installations. The first appendix is a 
highly readable discussion of the state of the art circa 1964. Some 
of the material is now out of date, but on the whole, it is the best 
general statement of how the computer is being used in school set- 
tings. Those who have feared that the introduction of the computer 
into an educational setting heralds a brave new world will find that 
their fears have no basis in fact—at least so far. Application of 
EDP methods to date have been limited to essentially accounting 
and housekeeping functions. In no case, have EDP methods touched 
the central nervous system of education—the teaching-learning 
process. This is well documented in the appendix describing reports 
of operating EDP installations, Summaries of these reports include 


this reader, too often 
administrators charged wi 
seem that as long as administrators, 
are for the keeping of records and 


ing is not that EDP methods are 
ministrative tasks, but the fact th: 
for little else, 


As can be gathered, this reader 
amount of progress that has been ma 


ae kward! This is hardly the 


BOOK REVIEWS 783 


The above comments are not to be interpreted as being critical of 
the work of Goodlad and his associates. On the contrary, the au- 
thors are to be commended for having exposed such a deplorable 
situation. It remains, however, that the monograph offers only the 
barest introduction to the use of EDP methods in education. The 
knowledgeable person will find little of substance or interest here 
and will have to look elsewhere for new ideas about EDP applica- 
tions in education. 

Three of the authors of the present monograph (Goodlad, O’Toole, 
and Tyler) have prepared a book, Computers and Information 
Systems in Education, which is to be published by Harcourt, Brace, 
and World in September, 1966. Some of the material is drawn from 
the present monegraph. 


REFERENCE 
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Sciences. Englewood Cliffs, New Jersey: Prentice-Hall, 1962. 
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University of Southern California 


Fortran Programming (II and IV) by L. Dale Harris. Columbus, 
Ohio: Charles E. Merrill, Inc., 1964. Pp. x + 146. 


This is a paperback version of Harris’ Numerical Methods Using 
Fortran reviewed previously by this reviewer. The paperback con- 
sists of four chapters and the two appendixes from the earlier book. 
There are some additions in the form of new footnotes and added 
materials, but usually pages are identical in the two texts. 

Thus, the book is designed to be a programming text. The readers 
who have struggled through the technical manuals and reference 
manuals of the various computer manufacturers will find Harris a 
delightful change. The book will be a good teaching document. 

Harris, who is an engineer, is not so concerned with storage 
problems as are many behavioral scientists. For example, no men- 
tion is made of chain jobs and temporary tape storage, the use of 
which the reviewer has found quite helpful. However, the use of 
COMMON and EQUIVALENCE statements for storage efficiency 
is adequately explained. The reviewer would prefer to see the 
storage problem as an added separate chapter. Hopefully, the re- 
viewer’s concerns about storage will soon become archaic. But for a 
while we shall have storage problems; for example, we frequently 
need ways to squeeze N X N matrices into storage without needing 
to reduce the sample size. 

Harris’ book should become widely accepted as a text for pro- 
gramming classes because of its readability. Competent program- 
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mers might find the appendixes valuable, but probably will prefer 
to use the manufacturer’s manuals. 


W. L. BasHaw 
University of Georgia 


Numerical Methods Using Fortran by L. Dale Harris. Columbus, 
Ohio: Charles E. Merrill Books, Inc., 1964. Pp. xii + 244. 


Harris attempts “to provide a suitable marriage” of computer 
programming and numerical methods. The behavioral scientist 
might be tempted to ignore his work, since it is directed to engi- 
neering and the natural sciences. However, the reviewer believes 
that the computer-oriented behavioral researcher will find the docu- 
ment quite valuable. 


gramming in Fortran II and Fortran IV languages. As a program- 
ming text, the book is excellent, although for this purpose an in- 
structor might prefer Harris’ shorter paperback. (See the review of 
Harris’ Fortran Programming II and IV by this reviewer.) 

Harris attempts to teach basie programming concepts in his first 
three chapters plus his fifth chapter. Of special interest is his chapter 
on iteration (Chapter 3). By treating indexing and DO loops in the 
context of the numerical method of iteration, Harris teaches the 
programming techniques and also shows the power of iteration as a 
numerical tool. He reserves many technical considerations for two 
large appendixes which should help the student focus on basics. 

The remaining chapters deal with numerical analysis techniques. 
The topics are interpolation (Chapter 4), differentiation and inte- 
gration (Chapter 6), solution of ordinary differential equations 
(Chapter 7), finding roots to equations (Chapter 8), and the simul- 


computer to solve complex problems through 
error. The applied sci- 
alytic solutions are not 
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The behavioral scientist will find no discussion of statistical 
problems and their solutions. However, included techniques have 
definite applications to the solution of some simulation and Monte 
Carlo problems, and obvious applications to some of the sticky 
statistical problems that arise from time-to-time. 

In summary, Harris’ text will have its largest audience in the 
natural sciences. The behavioral-scientist-programmer will have 
no need for much of the book, although the learner or beginner will 
find it well-written and highly understandable. 


W. L. BASHAW 
University of Georgia 


The Proceedings of the Ninth College and University Machine 
Records Conference by Don Morrison (General Editor). Mono- 
graph 4, Educational Systems Corporation, 1964. (No place of 
publication is indicated on the Monograph.) 


This paper bound volume contains all but three of the papers 
presented at the annual Machine Records Conference, held at 
Texas A & M University in 1964. There are a few general papers, 
beginning with the keynote address by Jack Woolf, dealing with 
data processing philosophy. Other papers concentrate on more de- 
tailed subjects, but again in a generalized sense, such as library 
circulations, information retrieval, and “Where to—By 1984.” The 
other papers, and by and large the majority, present data process- 
ing “experiences” in certain selected and discrete areas, such as 
data transmission, accounting, budget, accounts receivable, payroll, 
and class scheduling. 

The College and University Machines Record Conference should 
be praised for covering, quite competently, some aspects of data 
processing in education; however, the reviewer must take issue with 
the resultant document. The book is an indication of the present 
state of the art, and by including certain topics and excluding 
others is also a statement of philosophy of educational data proc- 
essors. The areas covered represent the “business” aspects of edu- 
cation. Although there are some comments on instruction, i.e. teach- 
ing data processing, or teaching by computer, these topics are not 
considered in very much detail, nor do they seem to be included as 
valid areas to be discussed at this conference. It seems strange, 
and probably remiss, to ignore that part of education exemplified by 
student learning. There is a paper presented on “Bid-Matching for 
Panhellenic Associations,” but none on the use of data processing 
for the benefit of instruction. There are a few sentences referring 
to testing, but no systematic treatment about evaluation procedures 
for educational product control. There are discussions about audit 
trails in the business and management areas of educational institu- 
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tions, but there is a complete lack of discussion of audit trails for 
student learning. The topics which are covered are well done, but 
there seem to be surprising omissions. 

Several statements were found about which argument can be 
raised. For example: “teaching by computer .. . can only be used 
for factual subjects. Interpretations, opinions, and theory cannot 
be taught by the computer” (p. 43) ; the presumption on page eight 
that improvement in instruction results from increasing faculty 
salaries; the decision that students should have choice of class sec- 
tion (p. 237); and finally, “The advent of serious interest in data 
processing in public schools is relatively recent” (p. 162). The defi- 
nition of “recent” seems questionable, since the author of this 
critique wrote “Automation in Education,” published in the Jour- 
nal of Machine Accounting in 1958! 

The strong points in the papers include the prevention of dupli- 
cation of data collections, fewer forms for students to fill out, more 
rapid collection and distribution of data, and the increased ability 
to obtain data prior to decision making. The papers fail, however, 
in fulfilling the hope presented in the opening address, i.e. “Educa- 
tional objectives, values, and philosophies must take prece- 
dence. . .” (p. 2). The papers do not fulfill this obligation. 


RICHARD E. SPENCER 
University of Illinois 


The Prediction of Academic Success by D. E. Lavin. New York: 
Russell Sage Foundation, 1965. Pp. 182. $4.00. 
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other predictive factors. In short these terms actually refer to 
the inaccuracy inyolved in predicting academic performance 
from ability measures alone.” (p. 25) 


After some excellent diagrams and figures illustrating the types of 
research studies, findings are presented in four broad categories: 
intelligence and ability factors, personality characteristics, socio- 
logical determinants, and socio-psychological factors. On page 107, 
in summing up cognitive style, the author attempts “an intuitive 
factor analysis” which comes up with six variables associated with 
academic performance including social maturity, emotional stabil- 
ity, achievement motivation, cognitive style, achievement via con- 
formance, and achievement via independence. Had the author re- 
searched a little more carefully, he would have discovered that the 
reviewer anticipated him by five years by coming to the same con- 
clusions as a result of an even more extensive analysis in the Jour- 
nal of Counseling Psychology (Summer, 1960). As the author ac- 
knowledged, the book is an outcome of a report presented to the 
Foundation in 1959, and there are few citations of later date. Five 
years of lag time in a report which attempts a global summary of 
this area is excessive. Certainly this delay detracts from the book’s 
value. 

The book is to be praised in providing some structure and order 
for the welter of studies in the achievement area. It would have 
been helpful if the bibliographic section could have been arranged 
in an order which would make it more possible to look up studies 
alphabetically. The author is usually very careful about reporting 
research results accurately. In reporting the Getzels and Jackson 
study (p. 141), however, he makes the error of stating that teachers 
preferred the high intelligence to the high creativity students, 
whereas a more accurate reporting of the findings would have 
stated that whereas the teachers preferred the more intelligent stu- 
dents over the generality by a significant amount, they did not so 
prefer the creative ones. 

The reviewer agrees with the dust jacket which avers: “As a criti- 
cal guide to a significant portion of the literature, The Prediction of 
Academic Performance will be a highly important reference work. 
. . , Admissions personnel, guidance counselors, and school psy- 
chologists and administrators will find it useful... .” With a little 
more effort, it could have been much more useful, but it represents a 
fair, if delayed, start in an area which has cried for straightening 
out for a long time. What is needed now is for someone else to 
bring us up to date. 


Joun C. Gowan 
San Fernando Valley State College 
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Decisions for Tomorrow, Plans of High School Seniors After Grad- 
uation by Ralph F. Berdie and Alpert B. Hood. Minneapolis, 
Minnesota: University of Minnesota Press, 1965, Pp. 185. 


Ralph Berdie and Albert Hood surveyed all graduating seniors in 
Minnesota High Schools in 1961 to identify their post-high school 
plans and the conditions related to these plans. This study was 
both a repetition and an extension of the 1950 Minnesota study 
carried out by Ralph Berdie. The 1961 Minnesota survey and 
analysis of the data obtained from high school seniors are docu- 
mented and are described most carefully by the authors. 

As a comparative study of 1961 against 1950, a number of 
questions, as major objectives of the survey, were posed such as: 
(1) Did the proportions of after-high school pursuits change during 
the decade? (2) To what extent did proportions of high ability 
students making these plans change from 1950 to 1961? (3) What 
are the relationships between ability and these plans? (4) What are 
the relationships of socioeconomic, cultural, and family conditions 
to students’ post-high school plans, and how have these relation- 
ships changed over the eleven years? (5) What are the relationships 
between personal attitudes and values and the after-graduation 
plans of high school seniors? 

The last question posed marked the most significant change in 
the 1961 questionnaire from the one used in 1950. Twenty-five 
items aimed at obtaining information with regard to attitudes and 
values were added to the questionnaire, 

The only finding of any significance for these factors seemed to 
be that college-bound students in comparison with others expressed 
more ease in social situations, less difficulty with authority figures, 
and more favorable relationships with the family. Thus, “attitudes 


elicited by the questions were directly rel i i 
fe Dh ie y related to social behavior 


The follow-up study 
Percentage of seniors 


BOOK REVIEWS 789 


The useful generalization about college population growth is 
firmly buttressed by the research done by the authors, specifically 
for Minnesota. They summarize their findings in Chapter 2, and in 
addition they suggest implications not derived from the data ana- 
lyzed. They found that financial assistance was less of a factor for 
a college goal in 1961 than in 1950. However, they state the impli- 
cation that some groups might choose college if financial aid were 
of substantial amount and if its assurance were known before high 
school graduation. 

There are a total of eighteen chapters plus a foreword and an 
index. The chapters are: 
1. Attitudes, Behaviors, and Plans of High School Seniors. 
2. An Overview, Results and Implications. 
3. National and Community Changes. 
4, Two Surveys Eleven Years Apart. 
5. A Follow-up One Year After Graduation. 
6. Ability and College Attendance. 
7. Effects of School and Community on Decisions For Col- 
lege. 
8. Personal Values and Attitudes. 
9. Socioeconomic and Ability Variables. 
10. Students Bound For College. 
11. Students Seeking Jobs. 
12. Plans of High-Ability Students. 
13. High-Ability Students from Workmen’s Homes. 
14. Girls Who Planned To Enter Nursing. 
15. Girls Who Planned To Attend Business College. 
16. Boys Who Planned To Attend Trade School. 
17. Boys Planning To Enter Military Service. 
18. Profiles: Who Chooses What Plan? 

While Decisions For Tomorrow is a careful and thoughtful reci- 
tation of the data analysis of a comprehensive survey, it should 
prove useful to a high school counselor when he considers the many 
factors which affect a student’s career plans. The statistical analy- 
sis employed in the study will certainly be of help to other educa- 
tional researchers who, hopefully, will undertake similar studies. 

When such studies are undertaken, this reviewer hopes that in- 
struments will be available for greater penetration into the cultural 
aspects of an adolescent’s background since the main items in the 
1961 Minnesota study—how many books in the home, what maga- 
zine subscriptions—are too limited. 


C. RUSSELL DE BURLO, Jr. 
Tufts University 
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The Dropout: Causes and Cures by Lucius F, Cervantes. Ann Ar- 
bor, Michigan. University of Michigan Press, 1965. Pp. viii + 
244. $5.95. 

Communicating the findings of a scientific opus to educators, be- 
havioral scientists, and civic minded laymen is a formidable task. 
Any endeavor to reach such a diverse audience through a single 
volume is to be lauded. Cervantes undertook such a venture and 
succeeded in recording and in interpreting the different phenomenal 
worlds of the high school dropout and of the graduate. 

The search for the etiology of early school leaving has swelled in 
recent years. No matter what the dependent variable has been, 
most researchers, have found differences between the early school 
leaver and the graduate, The network of suspected causes includes 
low intelligence, low reading ability, inadequate personality ad- 
justment, and low socioeconomic status. So many variables have 
turned up, though, that distinguishing cause and correlate from 
epiphenomenon has been no simple task. In bold contrast to the 
hunt and peck research is Cervantes book. The hypotheses guiding 
the study were derived from theory. The generalized hypothesis 
that dropouts and graduates are from different family structures 
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of conclusions rests upon the representativeness of the samples, 
upon the adequacy of the procedure, and upon the validity and re- 
liability of the data. The description of the method of selecting the 
sample is anemic. There is no way to know how ages, 1Q’s, and 
other variables had been distributed. IQ’s were obtained from 
school records. Which tests and how recently they were given, vital 
to careful matching, were not identified. Seventy per cent of both 
samples were from working class homes. Yet, how socio-economic 
status of the home was decided was not described. Was Cervantes 
justified in merging the samples from the six cities? No tests of the 
homogeneity of the sample were reported. Such tests are crucial. 
Were the dropout to be a product of a variable correlated with the 
family organization but not a family variable at all, then the null 
hypothesis is being rejected when it is true. That intercity differ- 
ences were observed in family kindred suggests that city as a source 
of variation deserves greater scrutiny. Finally, a substantial pro- 
portion of dropouts comes from rural America. These were not con- 
sidered in the sample. Yet, there may exist differences between ur- 
ban and rural dropouts. 

The dropouts, according to Cervantes, had difficulty in answering 
the questionnaire. In addition, interviews with dropouts and grad- 
uates were under markedly different circumstances. Developing in- 
ferences about etiology from post hoc interviews is suspect. What 
criteria the raters used, moreover, is not known. Nor is there any 
evidence that raters agreed with one another. The author acknowl- 
edges the lack of refinement of the nonparametric test he used. Yet 
his findings read as if family correlated parameters of school hold- 
ing power were discovered. 

The stated focus of the study was “the enveloping social system 
of the respondent as perceived and internalized by himself.” How 
may the reader be certain that the views expressed represent faith- 
fully the existential past or present? Does he know, moreover, 
whether the views presented would endure over time? Many drop- 
outs were out of work or in uncomfortable circumstances; many 
graduates were in favorable circumstances. Perhaps the perceptions 
of the past were determined by these immediate circumstances and, 
hence, may only be ephemeral. A more parsimonious statement of 
the findings would be that the dropouts differ from graduates in 
perceptions of family. This is not the same, however, as saying that 
the families of the two groups differ. When this distinction is made 
it makes it easier to understand two observations which baffled this 
reader: (1) parents of dropouts do not value schooling, (2) drop- 
outs ignore or reject parent values. While it may be argued that 
valuing is not a generalized attribute, worthy of consideration is 
the hypothesis that a “halo effect?’ for negativistic reporting oper- 
ated during the interviews and the data represent these negative 
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feelings of the dropout. The TAT is known to be sensitive to cur- 
rent environmental presses. The differences in aggression in the 
TAT stories observed by Cervantes, then, may also be triggered by 
the frustration or joys of the current circumstances. 

Dropouts and graduates, according to the author, represent polar 
opposites on several dimensions, e.g., troubled-calm; hostile- 
friendly; pawn-master; sensate-idealistic. This either/or conclu- 
sion may result, however, from an artifact of measurement; namely, 
protocol ratings were in nominal scales. One final point concerns the 
ex cathedra conclusions. Counseling, multi-track curricula, promo- 
tion of apprenticeship training, and utilization of resources similar 
to those under the Economie Opportunities Act were among the 
suggestions made to promote better use of human resources. While 
they were not the subjects of the investigation, they nevertheless, 
offer counsel to the schools, the government, and the community. 
They deserve investigation in their own right, but not endorsement. 

The limitations of the study are cited to offer a context to the 
reading of the book. The book is recommended to the intended audi- 
ence although its presumptuous title should be abjured. It will be 
enjoyed because the dialogue is scholarly without being pedantic 
and because the data are yeasty without being capricious. The 
book dispels many popular beliefs concerning the dropout. It 
showers the reader with new and poignant questions about educa- 
tion and social class. Above all, the book marks an important step 
in bridling the dropout, at least in research, 


Norman M. Cuansxy 
North Carolina State University, Raleigh 


Task Experience as a Source of Attitudes by Paul E. Breer and 
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and 6). This left open the possibility that the initial scales had 
evoked some sort of generalized response set. . . . With this pos- 
sibility in mind, we deliberately put our new neighborhood and 
family scales first on the assumption they were less likely to 
change than any of the others in the questionnaire.” (p. 218). 
“Once the results were in, (changes in subjects’ belief in God 
were in the opposite direction than had been predicted) it was 
relatively easy to think of reasons why it should have worked 
this way. In fact, it soon got to the point where we found it hard 
to explain why we had ever predicted the opposite in the first 
place.” (p. 174). 

“There were several considerations involved in planning this 
final experiment. First of all, we were anxious to do a single 
study which would incorporate all the major elements in our the- 
oretical scheme. These include behavior, situationally specific 
orientations, and the two kinds of generalization (lateral and 
vertical).” (p. 214). 

“This was one innovation, i.e., using different tasks for the two 
different treatment groups. In addition to this change, we decided 
to use more than one task in each of the two treatments.... 
The reason for the change was simple. We felt intuitively that 
a task experience encompassing a variety of similar tasks would 
be more likely to lead to attitude change than an experience 
based upon a single task, as in our previous experiments. . . .” 
(p. 140). 


In reading this book, the reviewer kept on harking back to an 
early paper by LaPiere (1934) which indicates (substantiated in 
subsequent researches) that pencil and paper assessments of atti- 
tudes have little or no relevance to behaviors displayed in real life 
settings. Breer and Locke’s basic assumption in the testing of their 
theory of the relatedness of attitudes to task experience seems to be 
that the questionnaire responses by subjects regarding their atti- 
tudes validly represent the attitudes they evidence in natural set- 
tings. This reviewer has grave doubts on this score. He also has 
grave reservations as to the impact of a delimited, experimental 
four hour session involving experiences with several laboratory- 
type task experiences on significant values held by subjects in rela- 
tion to religion and God, group versus individual effort, and equal- 
itarianism and authoritarianism. The authors, however, present 
statistical data which indicate such experimental sessions do indeed 
have an impact on value systems of the subjects and the impact- 
influence lasts—as measured by a questionnaire—at least more than 
a week or two. 

The theory propounded and to some extent tested in the seven 
studies (one in a naturalistic and six in a laboratory setting) re- 
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ported in this book is that an individual’s preferences, ideas, and 
values are derived in large measure from working on tasks and that 
these cathectic, cognitive, and evaluation orientations are trans- 
ferred to educational, religious, recreational, and other settings 
through both lateral and vertical generalization effect. Task is de- 
fined (p. 17) as “a piece of work to be accomplished.” This defini- 
tion, however, seems to ignore a number of “tasks” e.g., taking the 
family on a picnic, using the cards in one’s hand to make as many 
points as possible, hitting a golf ball 200 yards down the fairway, 
and scaling a rock-face that have other purposes than accomplish- 
ing a “piece of work.” The bifurcated psyche (pleasure and affec- 
tion) and socio (task-based) motivation of human actions and in- 
teractions hypothesized by Helen Hall Jennings (1949) and Her- 
bert T. Thelen (1960) is ignored or denied by the point of view 
expressed in Breer and Locke’s book. 

The authors discourse with skill and cogency on the issue of the 
types of generality (pp. 256-265) which an adequate theory should 
demonstrate, the contaminating influence (pp. 262-263) of the de- 
mand characteristics of the laboratory situation, and the use of con- 
trast in experimental design (p. 261). However, the generalizability 
of the Breer and Locke theory regarding how attitudes are devel- 
oped is called into question for this reader by the authors them- 
selves when they assert (p. 19) that “children take on the beliefs 
and values of their parents.” This apparently is without benefit of 
task experiences. If it can occur with young persons, why cannot at- 
titude development similarly occur with older ones? The assertion 
(p. 173) that “The experiment was designed to change subjects’ be- 
lief in God, and this is precisely what did happen” seems hard to 
swallow. The independent variables impinging on the subjects to 
achieve this “change of belief in God” comprised: pre-post ques- 
tionnaire administrations over four hours of time; the “tasks” of 
solving J1g-saw, miniature tower building, prediction and matrix 
problems in either an individual or group context; and being paid 
eee ce ain a i the time and effort invested. 
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Despite these seeming lapses, the authors retain their forthright 
and focused stance in pointing out: “. . . it remains questionable to 
what degree these tasks are representative of the broader popula- 
tion of tasks found in less contrived settings” (p. 260) ; that, having 
left the laboratory, they will “be hard put to demonstrate that job 
experience is responsible for attitude formation” and “that attitude 
change in the direction predicted by our theory could be explained 
in terms of socialization” (p. 269). However, they reaffirm their 
concern in attitude change that is not dependent on verbal com- 
munication but that which can be attributed “to the task experi- 
ence itself.” To test and explicate this position, they emphasize that 
a way has to be found to keep_task experience and persuasion 
separate. Their conclusion is: “There would appear to be no simple 
solution that is applicable in all situations.” 
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The Social Science of Organizations: Four Perspectives by Harold 
J. Leavitt (Editor). Englewood Cliffs, N. J.: Prentice-Hall, 
1963. Pp. ix + 182. 

This book consists of papers prepared by four of the 24 partici- 
pants at the two-week Seminar on the Social Science of Organiza- 
tion held at the University of Pittsburgh and supported by the 
Ford Foundation. 

The participants were divided into four smaller groups, each 
working on a major organizational problem. One member of each 
group was commissioned to prepare a perspective on the problem 
which would represent his initial position sharpened and tempered 
by his two weeks of interaction with persons of other views. Thus, 
the four perspectives were never intended to be balanced, exhaus- 
tive, or reportorial in nature, but they are incisive and competent, 
and they do reveal the multidisciplinary nature of organizational 
social science. Of the four authors, George B. Strother is a psychol- 
ogist; George Strauss, an applied anthropologist; Henry A. Latané, 
an economist; and David Mechanic, a sociologist. 

Strother discusses some of the problems in the development of a 
social science of organization by beginning with an excellent his- 
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torical overview. Strother identifies five perspectives or models of 
organization: ethical models, models which were used by Plato and 
which have survived Machiavelli to continue to the present; inter- 
relational models, used in the Hawthorne studies of the 1920’s and 
in the more recent laboratory studies of social psychologists with 
small groups; functional models, which have found sympathy with 
sociologists, and which define the problem in terms of group pur- 
pose and the unit in terms of the aggregate of individuals consti- 
tuting the organization; structural models, either in the rational 
form of Weber’s ideal bureaucracy or in the classical form of line 
and staff and departmentation; and mathematical models, asso- 
ciated with the names of Simon, Von Neumann and Morgenstern, 
and others. Strother then proceeds to a delineation of the logical 
problems of a social science of organization through emphasizing 
the specification of unit (individual, group) and of level of abstrac- 
tion (concept, relation, theory). He concludes by underscoring the 
current multidisciplinary trend in this area, 

Strauss considers certain aspects of power-equalization, the re- 
duction in power differential between supervisor and subordinate. 
The author criticizes what he refers to as the personality-versus- 
organization hypothesis, the hypothesis that human motivation in 
industry is incompatible with the objectives of the organization. 
According to this hypothesis, once man’s physical needs are satis- 
fied, his needs associated with creativity, individual expression, and 
autonomy come to the fore. In the fulfillment of its routine tasks, 
the organization can force subversion of these needs, or it can adopt 
Power-equalization techniques with a view toward focussing these 
needs in the direction of organizati 
Strauss, noting that the hypothesis has been advanced by acade- 
micians, for whom self-actualization needs are strong, doubts the 
universality of these needs, By questioning the validity of the 
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lytic model, though this fact is quite unrelated to its adequacy as a 
simulation model. Latané considers a model of the ideal, not a 
model of reality, and this distinction does not emerge clearly in his 
chapter. 

Mechanic concentrates on the methodology of organizational 
studies. In large part, the author organizes methodological prob- 
lems in terms of the great debate between those two perennial 
antagonists, the theorist and the empiricist. On the one hand is the 
theorist with his emphasis on validity, his interest in substantive 
problems, his penchant for global studies, and his use of variables 
defined in terms of their practical or theoretical significance; in the 
other corner is the empiricist, who concentrates on reliability, sys- 
tematization, restricted studies, and variables defined by the data, 
perhaps through factor analysis. Regarding research strategy, Me- 
chanic favors a multidisciplinary strategy, which implies different 
viewpoints (psychological, sociological, economic) of the same or- 
ganizational problem, rather than an interdisciplinary strategy, 
which implies a coalescence of ideas and, possibly, an eclecticism 
which may be too tolerant of weak ideas. Mechanic’s chapter is 
marred only by his use of such terms as “accept the null hypothesis” 
rather than “fail to reject the null hypothesis,” and “negative re- 
sults” rather than “inconclusive results.” 

Each of these four perspectives bears the mark of its author’s 
discipline, and their existence within a single cover emphasizes the 
extent to which the organization has become a center of multidisci- 
plinary study. 

EDWARD LEVONIAN 
University of California, Los Angeles 


The Causes and Cures of Neurosis by H. J. Eysenck and $. Rach- 
man. San Diego, Calif.: Robert R. Knapp, 1965. Pp. xii + 318. 
$7.95. 


As stated in the introduction, “This book is an introduction to 
post-Freudian methods of diagnosing and treating neurotics.” If 
this sentence is taken as a statement of purpose, the contents of 
the book broadly overshoot the authors’ goals, and, in other ways, 
fall far short of surveying the myriad of theories and treatments 
which have been applied to human beings debilitated by emotional 
and behavioral disorders in the post-Freudian era. 

The subtitle, “An introduction to modern behaviour therapy 
based on learning theory and the principles of conditioning,” gives 
a clearer statement of purpose and content than either the title or 
introductory statement. As such, this book joins Eysenck’s two an- 
thologies and the recent books by Wolpe, Bachrach, Franks 
Wolpe/Salter/Reyna, Ullmann/Krasner, Krasner/Ullmann, and 
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Bandura/Walters in expressing not only dissatisfaction with the 
scientific status and effectiveness of traditional “depth” theories 
and therapies, but a more promising alternative—the principles and 
techniques of learning. 

To the methods of treatment derived from the principles of learn- 
ing, Eysenck has previously given the generic label, “behaviour 
therapy,” to distinguish them from “psycho-therapy,” and espe- 
cially, psychoanalysis. This is perhaps an unfortunate term, be- 
cause when accompanied by such statements as, “Get rid of the 
symptom (skeletal and autonomic) and you have eliminated the 
neurosis,” the term itself tends to suggest that the “behavior thera- 
pist” is interested only in minor, specific, observable motoric be- 
haviors of a superficial sort; that the client may be dehumanized, 
mechanized, or otherwise, and; that the application of these meth- 
ods still operates within a disease-analogy model of symptom-with- 
underlying-unconscious-cause. For the reader who is not troubled 
with such implications, or who overcomes his emotional reactions 
for sufficient time to read the entire text, these inferences will be 
largely dispelled, e.g. “behaviour” encompasses not only motoric 
behavior, but also autonomic or emotional behavior, and cognitive 
or ideational behavior; such distinctly human problems as social 
anxiety, obsessive thoughts, and hysteria are functionally analyzed 
and treated; term such as “cure” and “symptom” are primarily 
used for alliterative convenience, and certainly do not allude to an 
acceptance of a “disease model,” even by analogy. While an ex- 
tended discussion directed at the above questions would have en- 
hanced the comprehension of this book, an excellent discussion 
does appear in Ullmann and Krasner’s Case Studies in Behavior 
Modification, which should be considered as a companion volume 
by prospective users, 
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the real “meat” of the book, which consists of a review and com- 
mentary on the various methods and techniques of “behaviour ther- 
apy” and the wide range of problems to which they have been ap- 
plied. The difficulty in organization apparent in these chapters 
seems to stem from the inadequacy of current nomenclature in 
handling the variety of problems which human beings can find for 
themselves—and the lack of anything better to replace the current 
system than a broad two-factor dimensional classification. Thus, 
we find several chapters which are problem centered, organized 
around more-or-less traditional diagnostic categories, such as 
“Anxiety States” and “Hysterical Disorders,” with the variety of 
treatment methods which have been applied to them reported within 
each chapter. Then, a chapter devoted solely to “Avoidance Con- 
ditioning and Aversion Treatment,” with the variety of disorders so 
treated reported within the chapter. This is followed by a chapter 
on “behaviour therapy” with psychotics, and then three chapters 
on “Children’s Disorders” organized still differently. The difficulty 
in organization, however, should not overshadow the excellent 
content included within the chapters on treatment. They do provide 
a most compact overview of the published literature on the range of 
problems and types of treatment programs which have been at- 
tempted. 

The theory and assessment chapters, on the other hand, will in- 
sure that Eysenck maintains his preferred role as a controversial 
author. In the first table, for example, the list of ten “most impor- 
tant differences between psychotherapy and behaviour therapy” is 
likely to raise the hackles of most “behaviour therapists” as well as 
a few “psychotherapists.” In addition, the chapters on “Dimen- 
sions of Personality” and “The Biological Basis of Personality” 
present data from factor analytic and twin studies to support 
Eysenck’s theories on extraversion-neuroticism-psychoticism in 
such a way as to impress the naive reader that there is no ques- 
tion but that these are “truth.” Perhaps the authors’ most bother- 
some tendency in the theoretical chapters is to make some rather 
broad leaps between data and conclusions by means of semantic 
equalities, e.g., Pavlovian “excitation,” attributed to introverts, is 
equated with cortical “activation” or physiological “arousal.” Data 
on the relationship between “drive” and performance are then 
taken as support for introversion-extroversion hypotheses. 

In short, this book provides, in a single cover, a concise sum- 
marization of the research and theories of Eysenck and his follow- 
ers across a broad range of areas—not merely “neurosis.” Work on 
behavior disorders to 1964, from a learning approach, is compre- 
hensively described and supported with available data relating 
both to outcome and theory. The chapters on children’s disorders 
especially, appear to be the most comprehensive to date, and the 
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extensive bibliography should aid interested readers in selecting 
original source materials. Since the book is written as an introduc- 
tory text, it should not be considered either as a manual on tech- 
nique or as a book on methodology. Where previous training is 
sufficient to allow critical examination, it would seem suited to give 
a broad overview for introductory courses at the graduate or post- 
graduate level. The optimum use of this book would call for sup- 
plement by additional texts emphasizing methodology, and a more 
thorough examination of the theoretical assumptions of the “causes” 
of behavioral disorders within a social framework. 


Gorvon L. PAUL 
University of Illinois, Urbana 


Individual Differences in the Classroom by R. Murray Thomas 
and Shirley M. Thomas. New York: David McKay Company, 
Ine., 1965. Pp. v + 567. 


The complexities of the human organism are reflected in individ- 
ual differences which are staggering to the imagination. Basic to 
effective classroom instruction is the teacher’s ability to cope with 
these differences. In addressing themselves to this gigantic task the 
authors state: “The most persistent, discouraging problems faced 
daily by teachers are usually caused by the fact that pupils differ 
in so many ways. ... there seems to be no single modern textbook 
designed to aid [teachers] adequately with their problems in this 
realm, The purpose . . . [of this book] is to help fill this gap” (p. 4). 

Ih limiting the scope of their book, the authors elect to focus on 
(1) intellectual differences, (2) differences in specialized abilities, 
like those affecting success in music, art, and motor activities, and 
(3) psychophysical differences, such as variations in hearing, see- 
ing, and speaking, They specifically exclude group differences such 
as race, religion, and social class. 

In developing their major thesis, the writers have attempted to 
answer several questions that appear vital to classroom instructors. 

ls What is the nature or definition of this facet of the child’s or 
youth’s life? 
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Psychophysical Differences. Part I represents an attempt to build 
the case for the remainder of the book. Indicative of the lively 
writing style employed is a statement from Chapter 1: “Children 
are not created equal, nor do they become more alike as they grow 
older. Rather, by the time they enter school the inequalities among 
them—intellectually, physically, and in social behavior—have in- 
creased manyfold. As they move upward through the grades, the 
differences increase even further” (p. 3). Chapter 1, then, casts 
some light on the extent and importance of differences, whereas 
Chapter 2 is aimed at enabling teachers to “. . . examine their be- 
liefs about individual differences so that they might form their 
values into a conscious, consistent system” (p. 34). A further stated 
purpose is to “introduce a philosophic orientation toward individual 
differences on which subsequent chapters are based.” 

Part II deals with various techniques of appraising individual 
differences, their range, and various administrative and methodo- 
logical problems encountered. Considered in Part III are those apti- 
tudes which seem to have little, if any, relation to those required in 
traditional academic pursuits. Included are differences in the 
graphic arts, those related to music, and those involving other types 
of “great and fine muscle coordination.” Finally, in Part IV, em- 
phasis is placed on psychophysical deviations. Included are vision, 
hearing, and speech disorders. Also treated are crippling conditions, 
such as cerebral palsy, epilepsy, heart conditions, disfigurements, 
malnutrition, and diabetes. 

A noteworthy feature of this book is the authors’ superb ability 
to come to grips with highly controversial issues. Instead of avoid- 
ing or glossing over such issues, they develop arguments for and 
then arguments against the issues and finally indicate the pre- 
ferred courses of action from the standpoint of the classroom teacher. 
This reviewer was impressed with their ability to make all points 
of view seem appropriate under given conditions. This agility is 
demonstrated in their discussion of the current controversy with re- 
spect to the single or unitary-factor theory of intelligence, as op- 
posed to the group or multiple-factor theory. “Then what is intelli- 
gence really, a single capacity or several clusters of abilities? The 
truth of the matter, from the standpoint of classroom applications, 
seems to be a combination of these two theories” (p. 58). Again, in 
Chapter IV, when the subject of homogeneous and ability grouping 
is treated, the writers avoid antagonizing groups by giving condi- 
tions under which ability grouping works best. 

This writer is hard pressed to find major weaknesses in this in- 
valuable contribution to an understanding of the myriad of differ- 
ences among growing, developing children. Obviously the writers 
were unable to treat, in depth, many of the problems. It seems that 
they have succeeded in pulling together in one volume many aspects 
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of the problem which have been widely scattered throughout the 
professional literature. They have focused attention upon many 
areas which tend to be neglected in other sources. This especially 
applies to so-called psychophysical deviations. 

In a book of this kind the writer must choose those problems 
which seem most important. It is almost impossible to deal with all 
aspects of such a diverse area. This reader did note what he con- 
siders to be two rather serious omissions. First, the problem of 
creativity was totally ignored. It would seem that the writers could 
have at least cleared up many misconceptions relative to the rela- 
tionship between academic aptitude and creativity. 

Another serious omission is the problem associated with being 
left-handed. Since almost 10 per cent of our youth must “limp along 
as best they can in a right-handed world,” this reviewer is per- 
plexed as to the reasons for this omission. It should be noted that 
this problem receives relatively little attention elsewhere. 

Another shortcoming of the book seems to be associated with the 
authors’ stated purpose of assisting the teacher in formulating his 
values into a “conscious, consistent system.” While it follows that 
serious consideration of the problems to which this book is ad- 
dressed should contribute to such a lofty purpose, this reviewer feels 
that the task has not been completed. Perhaps a culminating chap- 
ter, designed to help the reader make the desired connections, 
would have been most useful. It has long been known that the 
transfer value of learning is directly related to the extent to which 
the instructor teaches for this transfer. Accordingly, it seems in- 
cumbent upon the writers to lead the reader in the desired direction. 

It should be emphasized that the shortcomings are few and that 
they apply almost exclusively to areas of omission. The authors 
have struck at some of the most basic practical problems of teach- 
ers, Indeed if this book were read carefully by most school practi- 
tioners its impact would be marked. Many professional educators 


should find the book useful in a variety of teacher-preparation ex- 
periences. 


KennetH H. Hoover | 
Arizona State University 


Identification in Child Rearing by Robert R. Sears, Lucy Rau, and 


Richard Alpert. Stanford, California: iversi 
aegis ord, California: Stanford University 


oo a the study reported in this book is the interrelation- 
p hi -rearing practices and the behavior of four-year-old 
i ER he major dynamic is the process of identification. Iden- 
ification is presented first in the classic Freudian fashion begin- 
ning with anaclitic identification and subsequently developing into 
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defensive identification. The entire process of identification in the 
child was theorized to be a function of the child-rearing practices 
used by parents. 

An early problem faced by the researchers emerged from the dif- 
ferences between a descriptive theory of personality and a be- 
havioral theory which had predictive potential. It was necessary to 
trauslate the Freudian concepts of anaclitic identification and de- 
fensive identification from the traditional, descriptive psychoana- 
lytic theory into a dynamic behavioral theory which provided the 
potential for generating testable hypotheses. The data to test the 
hypotheses were collected through behavioral observations: through 
questionnaires to teachers and parents; and through interviews 
with children, with teachers, and with parents. The analysis of 
the data was confined primarily to tests of differences between 
means either through a critical ratio or ¢ and to the computation of 
correlations among the scales. 

The study is in the traditions of the careful field studies of child 
development in which several of the authors have previously partic- 
ipated. The interpretation is based less upon a carefully delimited 
mathematical experimental design than upon the application of 
some statistical models to test certain ideas about field activities in 
the child-rearing interactions. As a result, in some cases, the inter- 
pretation of significance was concerned more with whether any 
support for the idea could be detected than in terms of a rigorous 
theory of the reliability of statistics. The authors, at times, are 
quite willing to conclude that the differences observed in the data 
favored their analyses, even though the statistics were somewhat 
short of commonly accepted standards of statistical significance. 

In the development of the theory, the authors posited that de- 
pendency would be an important factor in the motivational system 
of the child and from it would emerge the functions of identifica- 
tion. The data did not wholly support their contention. In ex- 
amining the antecedents of the relationship between child-rearing 
practices and the development of identification, the authors reached 
the conclusion that initially, children of both sexes adopt feminine- 
maternal patterns of behavior. Boys, subsequently, at some point in 
the first three or four years, begin to alter their behavior towards 
the traditional masculine role. The activities of the father seemed 
to be insignificant in the development of the generalized male sex 
role of the son. Little attention is given to the commonly observed 
attachment of daughters toward fathers at about the same age. Con- 
trary to traditional theoretical assumptions, if the daughters also 
evidence a shift toward the masculine role at this time, their task in 
establishing their final generalized female sex role would be the 
more difficult. The boys would pass through a two-stage process by 
identifying first with the maternal role and subsequently shifting 
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to the masculine or father role. In contrast, the girls, also beginning 
with primary identification with the maternal role, subsequently 
would shift toward identification with the male or father role, only 
to have to shift their identification back to a feminine orientation. 
Thus, for girls a three-stage process is presented in contrast with 
the two-stage process of identification for boys. 

In general, the notion of anaclitic identification which develops 
very early in a child’s life, received substantiation. In contrast, 
however, the subsequent defensive identification received little, if 
any, support. The notion of a generalized trend of defensiveness 
was not supported. 

The book is an outstanding example of a carefully designed and 
executed field study. It is based upon a series of carefully developed 
theoretical postulates. Observation and measurement procedures 
were devised specifically for the problem at hand. The processes of 
data collection relied heavily upon framed interviews and well- 
trained observers. The resulting data were probably far more spe- 
cific and useful to the problem than they would have been had more 
standardized or traditional measures been used. 

The resulting mass of data was almost overwhelming. The last 
third of the book presents the data-gathering instruments and dis- 
cusses some of the problems met in collecting and processing the 
data. A total of 566 variables was coded: of these 246 were parental, 
and 310 concerned with the child. Some of them proved unac- 
ceptable or unusable. The final report concerns itself with 373 
variables: 184 parental, 189 child. For most of the variables, the 
total V used in the correlations was 40. In some cases, because of 
incomplete data, the N was somewhat smaller. 

It is difficult to avoid speculating on whether or not a more care- 
fully developed research design and the application of more exotic 
statistical technique could have added significantly to the findings 
in the book. However, at the same time, the converse speculation 
pers be avoided. Perhaps the results are more significant because 
ae Er gathered under actual field conditions than if the study 
ma EE eat in a more controlled, laboratory setting. To 
AATRE, way, a more coherent design for the generation of 

ale ight have detracted as much from as it could have added to 
P iy y. Even though a sophisticated design would have been more 
cs eo alte the currently accepted standards for educational 
Bey e ultimate contribution might have been considerably 

The stud 


dously antes an excellent example of an attack upon a tremen- 


; problem. The complexities of the reality were not 
ee neo the care given to the theoretical ass or to the 

e na/ysis in spite of the fantastic number of variables with 
which the research 


ers were forced to deal. They were able to carry 


ity n 
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their work through to completion. The study suggests that many of 
the phenomena which are endemic to the developmental process can 
be effectively studied on a short term basis. It also demonstrates 
that a field study need not be conducted at the expense of a theo- 
retical formulation, but can indeed profit from a careful theoretical 
basis. 

Geratp T. Kow1tz 

University of Houston 


Developmental Patterns in Reactions to Frustration by Udai Nor- 
ain Pareck. Asia Publishing House (distributed by Taplinger 
Publishing Co.) , 1965. Pp. ix + 182, $5.50. 

While the title may lead the unsuspecting reader to assume that 
he is about to enlarge his understanding of some aspect of child 
development specific to frustration, he may find himself engrossed 
by the details of a project to translate an American text for use 
with children in India. Dr. Pareck, in a doctoral dissertation at the 
University of Delhi, has developed a children’s form of the Picture- 
Frustration (P-F) Study for use in India and this book is a sum- 
mary of his efforts in validating this procedure. The problems of 
altering the P-F illustrations to “Indianize” the situations, the 
difficulties of translating into Hindi American terms which have no 
Hindi counterpart, and the tryout procedures are described in de- 
tail. While this is not the first effort at importing a text from one 
culture to another, it is of historical interest because the author 
does not feel the need to write in the cold, detached manner that 
characterizes many technical writers. 

Whether there is more here than anecdotal material about a text 
being transformed for use from one culture to another is difficult to 
say. Saul Rosengweig, the creator of the original P-F Study, tells 
us in a foreword to the book that the sample of 1,000 children is 
larger than his original standardization. It is difficult to determine 
whether the sample is, as the author claims, representative of chil- 
dren in India from ages 4 to 13, rather than representative of 
children in school in Delhi. In view of the extent of illiteracy in 
India, the standardization group from nine schools may be atypi- 
cal of Indian children in general. This suspicion of atypicality is 
increased when it is considered that Hindi, the language of the new 
version, is spoken by slightly less than half the Indian population 
(World Almanac, 1966, p. 609). 

Dr. Pareck makes cautious claims for the validity of his version. 
Correlations of test scores with behavioral ratings by teachers were 
nonsignificant and differences between delinquents and the sample 
were nonsignificant on six of seven scoring categories. The author 
was more successful in demonstrating shifts in P-F scores as a re- 
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action to induced frustration and in a “regional analysis” of homo- 
geneity. Comparisons of scores from Japanese and American man- 
uals of the P-F with the author’s sample is presented as a study of 
cultural differences. No data relating to matching of samples on 
such categories as sex, educational achievement, intelligence level, 
or socioeconomic level are presented. 

Although the comments have tended to be critical of Pareck’s 
efforts, there is much of interest to test constructionists in this book, 
Not only is the description of the translation process of interest, 
but his reviews of the research on frustration and its measurement 
(Chapter 2) and of research with the P-F (Chapter 3) will also 
prove useful. The overwhelming majority of his references are to 
American sources and this is, in the reviewer's opinion, the real 
failure on the part of the author. Here was an opportunity to justify 
printing and distributing the book in the United States in the first 
place: a review of what Indian psychologists are doing in per- 
sonality assessment research. 


Puur HIMELSTEIN 
Texas Western College of the 
University of Texas 


Substrata-factor Reorganization Accompanying Development in 
Speed and Power of Reading at the Elementary School Level by 
Harry Singer. Riverside, California: University of California, 
Riverside, 1965. (Multilithed) Pp. i + 276. 


This impressive monograph, a final report of research supported 
by the U. S. Office of Education over a number of years may be 
classed as a descriptive, developmental study. This cross-sectional 
study of pupils’ reading behavior in grades three through six is 
based on substrata-factor theory. The theory attempts to explain 
the mental structure and dynamics involved in reading ability. The 
findings from this study, derived by sophisticated statistical tech- 
niques, appear to make significant contributions of interest to those 


concerned with the psychology of readi i i d 
a ae gy ing and with educational an 


Design 


in six elementary schools, and (c) statisti 


i ee ys gaa analysis, Significance of difference between 
wi-groups, and Principal components factor analysis. 


s Singer created espe- 
» Particularly the test on Matching 
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Word Sounds, a silent reading test of sounding out whole words. 

His interesting Syllabication test is designed to measure inciden- 

tally application of general principles. The substrata analysis em- 

ployed, an extension of the Wherry-Doolittle Multiple Test Selec- 

tion Technique, is a useful tool for investigating and discovering 

statistically hierarchial structure or psychological organization. 
Findings 

The major developmental hypothesis of this study was confirmed: 
A sequential development of a hierarchial organization of sub- 
strata-factors did accompany improvement in speed and power of 
reading. A reorganization of substrata-factors tends to occur in 
which first level substrata-factors at lower grade levels become sub- 
sumed within substrata-factors at higher grade levels. Certain sub- 
strata-factors, however, (a) Visual Verbal Meaning for Speed and 
Power of Reading and (b) Speed of Word Perception for Speed of 
Reading, are first level factors throughout the intermediate grades 
(p. 195). The general premise that speed and power of reading are 
separate but interrelated components appeared to be tenable at 
least as early as the third-grade level. 

The minor hypothesis concerning a transition or shift in audi- 
tory-visual dominance as the average pupil moved through devel- 
opmental stages in reading was also confirmed. Throughout the 
intermediate grades, visual processes were dominant for Speed of 
Reading. But both visual and auditory processes continued to op- 
erate in Power of Reading. There was, however, some evidence that 
auditory abilities and processes may have shifted from more pe- 
ripheral to more central determinants of Speed and Power of Read- 
ing. Concomitant with development of general reading ability, au- 
ditory and visual subsystems became more closely integrated and 
functioned as a more centrally mediated communication system. 
This integrated subsystem appeared to have an interfacilitory ef- 
fect upon improvement in speed and power of reading (p. 197). It 
was inferred that the auding domain entered into speed and power 
of reading as a more centrally-integrated subsystem as individuals, 
in general, improved in speed and power of reading (p. 194). 

Significant differences between means of known-groups such as 
boys and girls and fastest, slowest, most powerful and least power- 
ful readers were found. It is likely, however, that all the known- 
groups at grade levels three through six mobilize at least minimum 
amounts of word meaning and word perception factors in their 
general working systems. In grades three through six, within-grade 
level superiority in Power of Reading was associated with quanti- 
tatively more rapid development in almost all substrata selected 
variables. Within-group superiority among the most powerful read- 
ers at the sixth grade lay primarily in the word meaning domain 
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(Visual Verbal Meaning and Meaning of Affixes) (p. 192). Dur- 
ing grades three through six, the slowest and least powerful readers 
were approximately three grades behind the development of the 
fastest and most powerful readers. For the benefit of these “de- 
prived” children, Singer developed the concept of the usefulness of 
substrata diagnosis and evaluation from the statistically deter- 
mined model to investigate the working system of these children 
for attaining speed and power, and to plot parsimoniously remedial 
and developmental action, The teacher could plot and compare 
graphs for the individual child with psychographs (derived from 
mean standard scores) for his referent group at each grade level. 

Perhaps the most significant finding for those interested in educa- 
tional and psychological testing is that some variables in reading 
appear to undergo systematic organizational changes as individuals 
improve in speed and power of reading. The variables could have, 
therefore, quantitatively and/or probably qualitatively different 
validities. 

The term, “substrata validity,” coined by Singer, has been im- 
plicit in substrata analysis, but this monograph makes the neo- 
logism explicit. Essentially, substrata validity implies that the 
validity of a variable is a function of its context within an empiri- 
cal and theoretical framework. Within this framework, a variable 
can have a low validity relationship with a criterion several levels 
removed, but can have a high predictive validity for a subcriterion 
nearer in the framework. Most importantly, whatever its quanti- 
tative relationship, its validity can also be judged on a qualitative 
criterion of being necessary in the mental organization for reading. 
To illustrate, visual or auditory acuity shows little correlation with 
reading comprehension. But, either one is important for discrimina- 
tion and perception which, in turn are foundations for abstractions 
and generalizations within a particular modality, reading or aud- 
ing. 

Factor analytic findings were consistent with an observation con- 
cerning a typical shift in instructional emphasis toward word 
meaning and mediational processing systems as the average pupil 
improves in speed and power of reading. As the grade level rises, 
word meaning variables do increase their loadings on Factor I, 
Visual Verbal Comprehension which finally emerges at the sixth- 
grade level as the combined Audio-Visual Verbal Comprehension 
factor. An implication for instruction and testing, drawn from 


systematic differences that were found between visual and auditory 


word recognition subsystems in relationship to Speed and Power 


of Reading was the following: objectives in grades three to six 


might be differentiated with emphasis upon visual word recognition 


for speed of reading and emphasis upon audi i 
subsystems for power of reading (p. 195) ‘ chad ee 


` 
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The implications of this study for the development of speed and 
power in reading in the classroom are the following: (a) In gen- 
eral the working systems uncovered at the successive grade levels 
provide a basis for determining important subskills and for de- 
ciding what relative degree of emphasis should be placed upon 
them. (b) The developmental models of first level substrata varia- 
bles give curriculum and test makers an overview of the scope and 
sequence of factors which allow for individual differences in Speed 
and Power of Reading, grades three to six. (c) Because there are 
some factors common to the developmental working systems for 
Speed and Power of Reading, instructional emphasis on common 
factors would contribute to the development of both working sys- 
tems. Subsystems needed for shifts from speed to power might be 
developed, perhaps by alternating instruction from accuracy to 
speed. In planning a reading program it appears that other factors 
—for example, attitudinal, motivational—should be included, since 
not all the variance was accounted for at any grade level. 

In the reviewer's opinion, this research represents a scientific ap- 
proach to reading instruction that will take the discipline even 
further away from the “chest thumping, eyelid lifting” mode of 
reading diagnosis of the past. With the discovery of ways of measur- 
ing additional variables that enter into reading and with the accumu- 
lation of studies of the type reviewed here, the discipline will move 
even farther in the direction of furnishing a scientific foundation for 
diagnostic testing and reading. 

Sara W. LUNDSTEEN 
University of California, 
Santa Barbara 


Clinical Aspects of Remedial Reading by Clifford J. Kolson and 
George Kaluger. Springfield, Illinois: Charles C. Thomas, Pub- 
lisher, 1963. Pp. 146. $5.75. 


This very short book purports to be a “complete” program for 
setting up procedures and practices to help children overcome read- 
ing problems. Of its twelve brief chapters that deal with diagnosis 
of reading difficulties, their treatment, and the reading clinic, the 
three on diagnosis will be of greatest concern for the reader of 
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT. 

Here, the authors divide reading disabilities into two categories: 
(1) the primary or congenital defects the symptoms of which they 
say are word blindness, disorientation of letters in spelling and 
digits in calculation, inability to recognize letter configurations, 
and right-left disorientation, and (2) the secondary or acquired 
defects the symptoms of which are slowness in synaptic transmis- 
sion (the passing of nervous impulses from one neuron to another), 
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visual and hearing defects, poor motor coordination, endocrine im- 
balance, low intelligence, poor home environment, poor teaching, 
and emotional problems. None of these so-called primary or second- 
ary defects is spoken of in general terms, however. Rather, a group 
of specific symptoms are given and these are then contended to be 
part of a generalized defect. How the several items in the secondary 
category are “acquired” is not explained. 

The approach to diagnosis given in the book is exemplified by the 
citation of authors whose works are considered a “must”: Delacato, 
Fernald, Herman (sic, see Hermann), Smith and Carrigan, Gilling- 
ham; and the centers for clinical internship deemed exceptional: 
Parker School in Chicago and the Orton Reading Center at Win- 
ston-Salem, North Carolina. 

The diagnostic procedures suggested follow this pattern. A child 
with reading disability is first given a two-part screening test. Here 
he (1) answers nineteen questions such as “Examiner touches sub- 
ject’s left hand. Which hand is this?” “Shows picture of left eye. 
Which eye is this?” and (2) writes dictation at an uncomfortable 
speed. If he “fails” these tests (norms for passing not given) he is 
said to have a “primary” disability, and then is given a mental 
test, a photographic test of his eye fixations, a silhouette completion 
test, visual and hearing tests, a figure drawing test (the Bender 
Gestalt is not used), the Kephart scale (walking a board, ocular 
pursuit, etc.), the Mills Learning Test, and diagnostic reading tests 
(a test of speech defects is notably missing). 

If the subject “passes” the above screening tests he is considered 
to have a secondary disability and is given only a mental test and 
diagnostic reading tests since “It is better, therefore, to dispense 
with an etiological diagnosis and proceed to a therapeutic diag- 
nosis . . .” because “In the diagnosis of most secondary reading 
disability cases it is impossible to determine the initiating cause or 
causes.” This is stated in spite of the number of readily ascertain- 
able aspects given as secondary causes. 

i Tt can be seen that this book breaks sharply with those descrip- 
tions of diagnosis usually found. Especially is this true with the 
emphasis given to neurological as versus psychological causes of 
reading disability. In the light of the existing research evidence one 
must view the premises for diagnosis given here as extremely tenta- 
tive, since relatively few reports have come out which relate reading 
problems to neurological defects. The evidence so far is that brain 
damage is seldom the cause of reading disability. As Arthur Gates 
says, the concept, of word blindness “can rarely, if ever, be applied 
to reading disability.” Likewise, little relationship has been shown 
between lateral dominance and reading disability. Correlations be- 
tween reading ability scores and visual tests also tend to be negli- 
gible. The relationship between hearing disability and reading, too, 


i 
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depends on the emphasis given to oral instruction. None of this 
evidence is examined in the book. 

Proof that these and many other research findings are wrong 
should have been an impressive part of the theory of diagnosis 
given in this book. Unfortunately, such an emphasis was not the 
case. Instead, Kolson and Kaluger chose to quote only the rela- 
tively few studies that give them some support, and to repress or 
ignore the weight of evidence against their diagnostic scheme. While 
this book erts basically in its omissions of evidence, it still is some- 
what useful as a challenge to the many more popular theories and 
practices in the diagnosis of reading difficulties. There is little doubt 
that such disputations are necessary for progress in this area. 


à PATRICK GROFF 
i San Diego State College 


ERRATUM 


The article “Kuder-Richardson Reliabilities of Classroom Tests” 
by Edward E. Cureton which appeared in the Spring of 1966 issue 
of this journal contained an error. aê on page 14, five lines above 

, the last should read ot. 

Dr. Cureton is indebted to Dr. Richard E. Spencer of the Uni- 

versity of Illinois for pointing out this error. 


EDUCATIONAL bh ace pat MEASUREMENT 
1966, 26, 815-824. 


pm AS AN “ERROR-FREE” INDEX 
OF RATER AGREEMENT 


JAMES C. NAYLOR anv E. ALLEN SCHENCK” ` 
The Ohio State oivan AoT 


Turn is a growing interest in the use of the multiple regression 
model as an analytical device for describing the strategies of indi- 
viduals in multiple cue choice situations. For example, the studies 
by Smedslund (1955), Uhl (1963), Peterson, Hammond and Sum- ` 
mers (1965), Summers (1962), and Schenck and Naylor (1965) 
are all instances of the application of regression analysis to the 
evaluation of performance strategies in probability learning tasks. 
Similarly, Naylor and Wherry (1965), Wherry and Naylor (1966) 
and Madden (1963) have used the multiple regression model to 
capture the strategies or policies of judges in attitude research. 

There is an important distinction, of course, between these two 
kinds of research investigations. The major distinguishing feature 
lies in the presence or absence of a criterion of “correct” behavior. 
In studies of decision-making, concept formation, and/or probabil- 
ity learning, the response of the subject to each stimulus can be 
evaluated in terms of its correctness, whereas in the case of attitude 
studies there is no such animal as the “right” response. 

In several recent methodological articles (Hursch, Hammond, 
and Hursch, 1964; Hammond, Hursch and Todd, 1964; Tucker, 
1964; and Naylor and Schenck, 1965), a number of performance 
indices based upon the regression model have been developed for 
examining the behavior of individuals in those situations where a 
criterion of correct behavior is present. However, as yet the applica- 
bility of these indices to choice behavior situations of a non- 
„criterion nature has not been forthcoming. Our purpose is to relate 
"he model and its performance indices developed in the criterion 
case to the non-criterion situation. 
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Consider the typical attitude assessment or rating situation 
where two judges are asked to evaluate the same set of n stimuli on 
some defined criterion dimension and where éach stimulus has k di- 
mensions which are potentially relevant to the judgment required of 
the raters. Such a situation is schematized in Figure 1, where 


X, = potentially relevant stimulus dimensions or cue values 
Gein ie Bs 

Y, = response of judge A. 

Y, = response of judge B. 

pa; = correlations over a population of stimuli between stimulus 
cue 7 and the response of judge A. 

ps; = correlations over a population of stimuli between stimulus 

_ cue 7 and the response of judge B. 
pi; = correlations over a population of stimuli between all pairs 


of k stimulus cues. 

Ba: = standard score regression weights for predicting Y4 from 
X; values. 

Bs; = standard score regression weights for predicting Y» from 
X; values, 


Y’, = predicted response for judge A based upon a least-squares 
regression equation relating his responses to the stimulus 
cue values. 


Y'a = predicted response for judge B based upon a least-squares 


regression equation relating his responses to the stimulus 
cue values. 


Assume Y4, Yz, X1,..., X;,..., Xz, to be random variables with 
means of zero and variances of one, that is, 


E(Y,) = E(Ys) = EX) =0 
and 


EY À = EY) = EÀ = 1. 
Further, define Y’4 and Y’; to be least- i 
ee 'B east-squares estimates of Y4 
Y,’ = 26,,X; and Yy = 2Bs:X. * 


10n all occasions, summations are over the k cue values, 


s 
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Stimulus 
Dimensions 


Judge A 
Response=Yq 


Judge A 
Predicted 
Response=Yq 


PaPa Pe =PYgYa 


Index of Agreement 
PnP iY 
Index of Matching 


Figure 1. Diagram of lens model showing the relationships among the stim- 
ulus cue dimensions and the judges’ responses. 


Then 
Y, = Y,' + Zi and Ys = Y; + Zs 
or 
Y, = Iba:X: + Za and Ys = 26s,X; + Zs, 
where 
E(Zs) = E(Zs) = 0 = E(Z.X) = E(Z2X,) 
and 


oy,” em lera” oh oz, =1= lors? ah ozs] sx Grs: 


One can then define the correlations in Figure 1 as 


p;; = E(X:X;) = correlation between stimulus cues 7 and j. 
pa; = E(Y4X,) = validity of cue 7 in determining judge A’s 


responses. 
ps: = E(Y,X,) = validity of cue 7 in determining judge B’s 
responses. 
E(Y, Y,” re y Tea 
Ppa = ae a E(YsY.’) sant ZBaiPat 


= multiple correlation of judge A, i.e., the degree to which 
one can predict his responses using all k cues. 
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pp = Eliae) L VEY) = VZBrps: 


Oya’ 
= multiple correlation of judge B, i.e., the degree to which 
one can predict his responses using all k cues. 


The last two coefficients have the well-known property that 


pa = oy, and pr = yy. 
Agreement between Judges 


Let us call the relationship between the responses of two judges 
(A and B) “response agreement” and define it in the same manner 
as “achievement” has been defined where the model was used to 
analyze performance in the presence of a correct response (e.g., see 
Hursch, Hammond and Hursch, 1964; Naylor and Schenck, 1965) ; 
thus 


Pa = E(Y 4Y») = response agreement between judges. (1) 
The correlation between residual scores is then defined as 


C= E(Z,Z»), (2) 
F2747ZB 

Equation (1) is the normal manner in which agreement between 
judges is defined, i.e., the correlation between their sets of responses. 
However, we are usually not only interested in the agreement be- 
tween what the judges actually do, but we are also interested in the 
agreement of their basic judgmental policies, These are not quite 
the same thing, Previously we have defined the policy of a judge in 
terms of “what that judge does” (Naylor and Wherry, 1964). This 
definition is no longer convenient—rather let us now define the 
policy of a judge as his least-squares regression equation. In other 
words, his policy is defined as our best prediction of what he will do 

(given our model), rather than what he actually does. 
An index of policy agreement, or matching, must thus be based 
upon predicted scores. Such a matching index is obtained using the 


correlation between the estimates or predicted scores of the judges 
as follows: 


pa = EEY) 


Oyg/Tyy’ 


= index of policy “matching”, i.e., the 
extent to which the judges’ policies 
are similar. (8) 


fi 
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If we say that a judge’s policy is represented by the regression 
weights of his prediction equation (e.g., Y'a = 3 BasX), then any 
measurement of policy matching between two judges should meas- 
ure the similarities between their respective regression weight vec- 
tors. One way to do this is to intercorrelate the two regression weight 
vectors or, as a substitute, intercorrelate the cue validity vectors 
(PROF technique of Wherry and Naylor, 1966).2 However, these 
methods are subject to possible criticism if the matrix of covar- 
iances between all of the cues is significantly different from an iden- 
tity matrix. For if the cues are not orthogonal, the meaningfulness 
of the relative magnitudes of either the regression weights or the 
cue validities must be questioned. (Hoffmann, 1962; Ward, 1962). 

On the other hand, pm can be interpreted in a very straightfor- 
ward manner as the correlation between the prediction equations or 
hyperplanes. Consequently, pm is clearly a measure of policy simi- 
larity. Whenever two policies are identical (that is, when Ba, = Ba 
for alli = 1,...,k), pm will equal one. Any deviations from Ba; = 
Axi, other than an exception to be discussed below, will make pm 
AR 

There is one exception to the above rule, and that is when Y’, is 
some linear function of Y’s, i.e., Y'a = bY’s + a. However, we 
know that 


E(Y,') = E(Ys') = 0; 


therefore, 
a=0. 
And since we know that 
E(¥,”) = p° and E(Y5") = ps’, 
then 


b=. 
PB 
The resulting relation is 
KA RUSTA 
Pe 


2 For a discussion of the relative merits of these two procedures see Naylor 
and Wherry, 1964. 
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And, it will also be true that 


Bas = PA Bai, 
PB 


Consequently, when pm = 1, we know that either the policies are 
identical or they are in proportion by a known constant. Whether 
two policies, related by the constant, ps/ps, are really different, is 
a matter of choice. Certainly, the only difference is in the consist- 
ency with which the judges adhered to their respective policies. 

Tucker (1964) gives us another useful interpretation of pm (or as 
he has named it, G). If we consider Y’4 and Y’s to be vectors with 
lengths pa and pz, respectively, pm is then the cosine of the angle 
between Y’4 and Y’z (see Figure 2). 


\ 
Length of Ya=lYal= p ve, 


Length of Ys=lYgl= 


Figure 2. Pictorial representation of predicted score vectors, Y'a and Y's, 
and their degree of similarity, pm, suggested by Tucker, 1964. 


It is interesting to note that Hursch, Hammond and Hursch 
(1964) did not mention pm. Instead, they discussed a term, 3d, as 
the measure of agreement between a subject’s policy and the policy 


(structure) of the ecology (which we have substituted with another 
subject or judge). 
Using their symbols, 


Zd = 2(8,; — Bui) (pes — Pri). 
Substituting with the present notation, we have 


Zd = ZBas aa Bsi) (pax card xi). 
This measure is clearly sensitive to any differences in policies and 
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can be shown to be directly related to pm. In order to see this, con- 
sider the equation developed by Hursch, et al., (their Equation 27, 
p. 55) to describe achievement or, in our case, response agreement: 


pac R ERIA, CVa-R)G-RyY. (8 


This can be rewritten in the notation used here as 


2 2_ >», 
pa = tee PE VT po) O) 
Another expression of pa has been developed using pm. From Naylor 
and Schenck (1965) we have 
pa = E(VuYn) = El(¥a' + Zs)(Y¥u' + Zs)] 
E(Ya'Yx') + E(Ya'Zs) + E(Ys'Zs) + E(ZsZs) 


= PmOva'Frn' + Cozsoze 
= pmpaps + OVAL — pè) — pr). (6) 
The difference between Equations 5 and 6 is the first term on the 
right side of the equality, that is, 


2 2 
— Xd 
pat py — 20 = papap. 


Therefore, 


Zd = pa? ae pa — 2pmpaps- 
Thus, Hursch, et al., have used a measure of policy matching which 
is simply the square of the distance between the two vectors Y’4 
and Y's (see Figure 2). 


Comparisons of pa and pm 
If we consider Equation 6 and make the reasonable assumption 


that the residual scores for the judges are random, i.e., H(Z4Zz) = 
0, then 


Pa = PmPaPa- (7) 

This relation states that response agreement is the degree to which 

the two judges successfully match their regression equations, 
weighted by the product of the intra-judge consistencies. 

pm is, a8 we have defined it, the correlation between the estimates 
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or predictions of the responses of judges A and B, that is, between 

7, and Y's. Thus it is necessarily the similarity between the two 
prediction hyperplanes or better yet, between the two policies un- 
encumbered by error. Consequently, pm is an “error-free” index of 
policy similarity or agreement. It is different from pa, response 
agreement, in that the latter is affected or attenuated by errors in 
prediction. This attenuation is measured by the indices, pa and pz. 
Equation 6 clearly shows that only when both policies are entirely 
predictable or captured, i.e., pa = ps = 1.00, will response agree- 
ment, pa, be as high as policy agreement, pm. Otherwise pa will al- 
ways be less than pm? 

An interesting parallel can be drawn here with measurement 
theory. All practitioners in testing and measurement are familiar 
with the concept of “correction for attenuation.” One can ask what 
the “true” relationship between two tests or variables would have 
been had there been no errors in measurement, The formula usually 
encountered is 


py.y, (true) = PYY, k 
PY. ¥:PYs¥s 


In a like manner, we can ask what the agreement between two 
judges would have been had they been perfectly reliable. The im- 
portant distinction here, however, is that unreliability is the failure 
to account for all of the judge’s variation with a linear combination 
of the stimulus cues, i.e., the degree to which that judge is “incon- 
sistent.” With this in mind it can be seen that 


pa(true) = py,r,-(true) = 74 
PrayaPya'Ya’* 

Now since p4 (true) represents the correlation between Y4 (true) 
and ¥’, (true), i.e., the correlation between a person’s true score 
and his predicted true score, it must be unity. Similarly pya'ya’ Must 
also be unity since it represents the reliability of a prediction equa- 
tion based upon the entire population of elements. pray, 18 not 1.00, 
however, since we still must consider error in the measurement of 
the judge’s responses, Thus, we have 


Pa esse Se! 2 LEE Pat= V pra. 
3 This statemen 


nee ii ea of bate hein that only positive or absolute values 
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This equation bears out the difference between the two types of un- 
reliability mentioned above, in that p's, which is a measure of the 
degree to which we can predict that judge’s responses from the 
stimulus values, is his reliability coefficient. 
A similar derivation can be made for judge B so that we can now 
define “true” agreement as 
Pa Pa 


pia V praraPysYs PaPa pic 
The result shows that if we approach pm from an error of measure- 
ment point of view, it can be defined as the agreement between 
judges corrected for attenuation. On the other hand, pm is also re- 
sponse agreement corrected for failure to capture, without error, 
the policies of the two judges. 

In conclusion, the use of pm as an index of rater agreement, 
either in place of or along with the more common index, pa, would 
appear to offer interesting possibilities. As an index of policy agree- 
ment it is “error-free” and indeed can be viewed as pa corrected for 
attenuation due to judge unreliability. 
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SIMILARITY ANALYSIS BY RECIPROCAL PAIRS 
FOR DISCRETE AND CONTINUOUS DATA 


LOUIS L. McQUITTY 
Michigan State University 


Tum similarity index was developed as a central technique in an 
early method of hierarchical pattern analysis, called similarity 
analysis (McQuitty, 1955). 

Similarity analysis has the advantage of being applicable to both 
discrete and continuous data. However, in its original form, it has 
the disadvantages of being complicated and laborious, sometimes 
requires iteration, and can lead to inconsistencies (McQuitty, 
1955). By application of a particular theory of types (McQuitty, 
1966) all of these problems can be solved, and a very simple method 
of hierarchical analysis can be developed, called Similarity Analy- 
sis by Reciprocal Pairs, applicable to both discrete and continuous 
data. 


Theory 


The theory says that every individual represents a succession of 
types, first an individual type, then types analogous to a species, a 
genus, a family, etc. As more and more individual types are classi- 
fied together to represent higher and higher orders of hierarchical 
types, the successive categories become better representatives of 
pure types, which exist only in theory. 

Individuals and hierarchical categories of the above kind are 
jointly characterized as typal representatives. 

Every typal representative at any level x is best classified at the 
next higher level if it is classified with the typal representative most 
like it at level x and if the two representatives are reciprocal, i.e., 
Typal Representative i is most like j, and j is in turn most like 7 
(McQuitty, 1957). 
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The Basic Equation 
Let: 


ij = any typal representative formed by combining the two 
typal representatives 7 and j from the next lower level, 
level z. 

k = any typal representative other than 7 and j from level z. 

Gi, = an index of association between 7 and k. 

Gi, = an index of association between j and k. 

q;;-, = an index of association between 7j and k. 


Then: 
Qij- = its, (1) 
a 
An Illustration 


In order to illustrate the method, it was applied to the data of 
Table 1, coefficients of correlation between people for the second 


? TABLE 1 
Coefficients of Correlation between People 
SSS. ooo 
B Cc D E F 
E ee ne F 
B 60 49 24 43 
Cc 60 62 46 51 
D 49 62 40 57 
E 24 46 40 67 
F 43 51 57 56 
i Daa from eee AD p. i: entries have been rounded from two to three 


through the sixth persons of a m. 
169). 

The coefficients of correlation were first converted to variances by 
Squaring them and are shown in Matrix 1 of Table 2. This is not 
essential in most cases but does help increase the accuracy in de- 
tails to the extent that variances represent better units than do 
Coefficients of correlation. 

The first step is to underline the highest entry in each column of 
Matrix 1, Table 2; it is 36 for Column B, 38 for C, 38 for D, and 
31 for each E and F, Then, select the highest, entry in the entire 
matrix; it is 38 and mediates between individuals C and D. In- 


atrix from Stephenson (1953, p. 
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a 


1 7190, fo SPUN U01702 4409 IY} woul pandwoy SIPUT UDDA fo sishjouy Apsnjruns y 
z TIJVL 


e IUW Z WEN T UPN 
TE æ 9% sI 
Fas ES A, es ce TE or 12 90 
wee Ie ye a TE 6 0% T ze OT BE LZ 
aa TE (ai a 6 6r we do 9% z BE dE 
oa x g aoa st %9 æ a s1 90 2 9% 
i a doa 4cm Gd & ic & Sa ps a 
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dividuals C and D jointly are accepted as a better representative of 
a hierarchical type than is either separately. They are joined as a 
type and placed in both Row CD and Column CD of Matrix 2, 
Table 2. 

Using the basic equation, the entry for both Row B—Column 
CD and Row CD—Column B is: 

Qcp-3 = Groene = oe = = = 31.50, rounded to 32. (2) 

The other entries of Row and Column CD are computed in an 
analogous fashion. The entries for the remaining cells of Matrix 2 
are taken from the corresponding cells of Matrix 1. 

The highest entry in each column of Matrix 2 is underlined and 
the highest entry in the matrix is chosen, 32 for Row B—Column 
CD and Column CD—Rovw B. 

Typal Representatives B and CD are joined to form a higher 
Typal Representative, BCD, and are entered as a row and column 
of Matrix 3. 


The basic equation is used to compute the entry for Row E—Col- 
umn BCD and Row BCD—Column E, 


Qsco-g = 2 E ETE Et toa 12.50, rounded to 13. 


(3) 

The entry for Row F—Column BCD and Row BCD—Column F 
is computed analogously. The other entries of Matrix 3 are taken 
from their corresponding cells of Matrix 2, 

The highest entry in each column of Matrix 3 is underlined, and 
the highest entry in the entire matrix is determined, 31 for Row E— 
Column F and Row F—Column E. Typal Representatives E and F 
are therefore combined to form a new type at a higher level, Typal 
Representative EF. This completes the analysis, However, for pur- 
pose of further illustration, we show how the similarity index for 
Row BCD—Column EF and Row EF—Column BCD is computed: 


Ch ee Qzr-Bep + Ap_zop 
2 


13 +24 37 
= 3 = 9 = 18.50, rounded to 19. (4) 
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A Critique 
All reciprocal pairs of any matrix could have been operated on in 
preparing a new matrix. For example, Matrix 1 contains two recip- 
rocal pairs, CD and EF. Each of these could have been combined 
to yield Matrix 5 as shown in Table 3. 


TABLE 3 
An Alternative Similarity Analysis of Matriz 1, Table 2 


BCD’ EF BCD EF 
B 32 12 BCD 18 
cD 32 24 EF 18 
EF 12 «(4 

Matrix 5 Matrix 6 


The entry for Row B—Column CD and Row CD—Column B is 
computed in the same fashion as it was for Matrix 2, Table 2, and 
likewise for the entry of Row B—Column EF and Column EF— 
Row B. 

The entry of Row CD—Column EF and Row EF—Column CD 
was, however, computed in a slightly different fashion; two applica- 
tions of the basic equation were involved: 


Qcp-2 = a = 18.50, rounded to 19. (5) 


Qop-r = 


26 + 82 op © 


Gcp-zr = 21416+2632 = 23.75, rounded to 24 (7) 


or 
Qcn-zr = nae = 24, (8) 
Stated in general notation, Equation 7 states, 
Qyj-n = Qir + On + Qir + tir, 9 


This entire equation can be generalized to: 


zen yem 
Oxy 
-aim (10) 


8, = 
z nm 
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Say = the original similarity index (McQuitty, 1955). 

z = Type z represented by Individuals 2,, £a, Za, +++ Nn. 

y = Type y represented by Individuals yı, yo, ys, *-* M. 

The difference in Equations 1 and 10 derives from a difference in 
theory. Equation 1 says that each type should be given equal weight 
when they are joined to form an hierarchical type of the next 
higher level. Equation 10, on the other hand, says that the two 
types being joined should be given differential weights in relation 
to the number of individuals in each of them. Equation 7 (a special 
case of Equation 10) gave the same results as Equation 1, because 
in the special case of Equation 7 each CD and EF were composed 
of two individuals and were therefore given equal weights. 

The theory of this paper recommends Equation 1; Equation 1 
was derived from the theory of this paper and has the advantage 
of being very simple to apply to both discrete and continuous data, 

An alternative theory of types could require differential weights 
in relation to the number of individuals in each of two types being 
joined to form a new type at the next higher level. An equation 
simpler to apply than Equation 10 can be developed for the pur- 
pose. 

Let: 


i =a type just realized by combining Types j and k; it will 
replace Types j and k in the next matrix. 

= a type composed of Individuals bi, ba, ba, +++ bne 

= a type composed of Individuals ¢,, Cz, Ca, +++ Cys 

= any other type realized earlier by combining Types x and y. 

= a type composed of Individuals dı, ds, da, +++ dp. 

y = a type composed of Individuals eis €a, Cg, ** t ene 

t = the similarity index between Types 7 and l. 

tı = the similarity index between Types j and l. 

% = the similarity index between Types k and l, 

Then: 


acre. 


ay, = Gn) + mlan) (11) 


n+m 
Differential weights are used for Type 7 in the above application 
of Equation 11; they are represented by n and m, showing the num- 
bers of individuals in each Type j and k which combined to yield 
Type i. Differential weights are not, however, used for Type l in 
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the above application of Equation 11; they were used earlier in the 
analysis when Equation 11 was applied to compute each ay and 
arı; they are represented by r and s. 

Similarity Analysis by Reciprocal Pairs can be used to analyze 
any matrix (even one generated out of chance) into statistical 
types, for every matrix has at least one reciprocal pair. Conse- 
quently, the investigator should be especially aware of the need to 
find other evidence before interpreting the results as indicative of 
substantive types. 


Summary 


This paper generates an especially simple method for analyzing 
both continuous and discrete data into hierarchical types. 
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THE RELATIVE EFFICIENCY OF TEST SELECTION 
METHODS IN CROSS-VALIDATION ON GENERATED DATA 


LESTER C. SHINE II 
The Ohio State University 


Tum basic problem in test selection is one of selecting a subset of 
predictors (tests), given a larger set of possible predictors, in such 
a manner that a measured criterion can be predicted as efficiently 
or more efficiently (due to high shrinkage), in the probability 
sense, from a weighted composite of the subset of predictors as it 
can be from the whole set of weighted predictors. In other words, 
the aim of test selection is to maximize prediction of a given cri- 
terion and minimize cost where cost is a factor. To accomplish this 
aim, two common approaches have been followed. These approaches 
are statistical inference of predictor weights based upon a single 
sample (Wherry, 1951) and the technique of cross-validation 
(Mosier, 1951). The cross-validation approach has been studied 
here because it is the more realistic in the practical sense and be- 
cause it overlaps with the statistical inference approach. 

Implicit in the problem of test selection is the question of dif- 
ferential weighting of predictors, because no test selection can occur 
without differential weighting. Optimal weighting effectiveness usu- 
ally occurs when there is considerable variation among the weights 
of a chosen weighting system (test selection method), when predic- 
tor intercorrelations are all small, and when correlations of the 
chosen weights with corresponding weights of other weighting sys- 
tems are small (Scates and Noffsinger, 1931). It can be shown, how- 
ever, that, under certain conditions, if the number of predictors is 
very large, the weighted composite predicts a given criterion equally 
well regardless of how the predictor weights are chosen (Wilks, 
1938). Further, the presence of suppressor variables in a battery of 
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predictors may cause ordinary test selection methods to break down 
during the selection process itself (Wherry, 1946). Thus, it seems 
that, in addition to the obvious variable of sample size, the vari- 
ables of number of potential predictors and homogeneity of po- 
tential predictors (as measured by size of test intercorrelations) are 
the more important variables to consider in an investigation of the 
relative efficiency of test selection methods in cross-validation. 

The most comprehensive investigation to date of the relative 
efficiency of test selection methods in cross-validation has been con- 
ducted by Perloff (1951). Perloff manipulated sample size in his 
work while holding constant the variables of number of tests in the 
battery (seven) and homogeneity of the tests (high). His results in- 
dicated that the Standard Score Unit Weighting method worked 
best for small samples, Regression Analysis was best for very large 
samples, and the Wherry Test Selection method worked best for 
medium to large samples. Other less comprehensive comparative 
studies of test selection methods have been done by Anderson and 
Fruchter (1960), Grimsley (1949), Lawshe and Patinka (1958), 
and Lubin and Summerfield (1951). The gist of previous research in- 
dicated that the Wherry Test Selection method, Regression Analysis, 
the Wherry-Gaylord Test Selection method allowing plus one or 
zero weights only, and the Wherry-Gaylord Test Selection method 
allowing plus integer or zero weights only were the best four meth- 
ods to study under the manipulation of the variables of sample size, 
Size of test battery, and homogeneity of test battery. Original 
sources for the Wherry Test Selection method and the two forms of 
the Wherry-Gaylord Test Selection method can be found in Stead, 
Shartle, et al. (1940) and Wherry and Gaylord (1946), respectively. 

In general, previous studies on comparisons of test selection meth- 
ods have been based on real data rather than computer generated 
data. Using computer generated data appeared to be better for the 
Present investigation for two reasons. One reason was that the char- 
acteristics of computer generated data can be precisely controlled, 
whereas real data cannot usually be controlled so precisely. The 
Second reason for using computer generated data instead of real 
data was that the size of the present investigation virtually pre- 
cluded the Possibility of using real data.1 


—— 
1 This article is b; 2 2 
J. Wherry, The Ohio State Uae gee ae eos done under Dr. Robert 
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Method 


Subjects 


The subjects used in this experiment were entirely fictitious. The 
computer generated data used were equivalent to 40,000 subjects. 


Apparatus 


The only apparatus used in this experiment was the computer 
facilities of The Ohio State University (OSU) Computer Center, 
consisting of an IBM 1620 Computer and an IBM 7094 Computer. 


Procedure 


A fixed factor, four-way Analysis of Variance with no repeated 
measures and a within cell n of 5 seemed to be an appropriate analy- 
sis to use in the present investigation. The four factors were Sample 
Size, Size of Test Battery, Homogeneity of Test Battery, and 
Method, denoted by N, S, H, and M, respectively. The levels of N 
were chosen to be 50 and 200. The levels of S were chosen to be 6 
and 12 (did not include the criterion). The levels of H were chosen 
to be Low and High, as measured by test intercorrelations. The 
levels of M were the Wherry Test Selection method, Regression 
Analysis, the Wherry-Gaylord Test Selection method allowing plus 
one or zero weights only, and the Wherry-Gaylord Test Selection 
method allowing plus integer or zero weights only, denoted by WT, 
RA,WG + 1,and WG +I, respectively. 

The basic data for the present study were randomly generated 
intercorrelation matrices having built in N, S, and H characteris- 
tics. These characteristics were built in through the use of the 1620 
Score Generator Program available at OSU. This program made use 
of a theorem in Factor Analysis which says that Z = AF (Harman, 
1960), where Z is the matrix of observed scores on each test, A is 
the population factor structure composed of factor loadings for 
each test, and F is the matrix of scores on each factor variable, with 
all factors assumed to be mutually independent of each other. 
The program randomly drew samples of pre-assigned size N of fac- 
tor scores from normally distributed independent populations, and 
then performed the matrix multiplication A times F to produce the 
matrix Z of observed test scores for each test. For data generation 
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purposes the criterion variable was considered to be the last test in 
the matrix Z. The tests were then intercorrelated to produce the 
observed intercorrelation matrix of predictors and criterion. 

Since the matrix A entirely determined the characteristics of S 
and H, and since S and H each had two levels, four population 
matrices A were required for data generation. The number of rows 
of A controlled S, with 7 and 13 being used (the last row was re- 
served for the criterion). The pattern of loadings of A controlled H. 
Restrictions were imposed that the true population multiple R? be- 
tween predictors and criterion range from .501 to .507 across the 
four A matrices and that the number of common factors be held con- 
stant at four for each of the four A matrices, with unique factors 
being included for each test. The four A matrices are presented in 
Table 1. 

The proposed 2X2X%2%4 ANOVA design with a within cell n of 5 
implied that a total of 320 appropriate intercorrelation matrices 
were required. After the necessary 320 matrices were generated, the 
four test selection methods under consideration were applied, ac- 
cording to the above ANOVA design, to the first 160 matrices, here- 
inafter called validation matrices, of the 320 matrices to obtain test 
selection weights. This phase was called the validation phase of the 
experiment and its sole purpose was to obtain test weights from each 
of the 160 validation intercorrelation matrices. Standard criteria 
for stopping selection of tests were used where applicable. The 160 
sets of test weights obtained in validation were then applied to the 
Temaining 160 intercorrelation matrices, hereinafter called cross- 
validation matrices, using the standard formula for a composite 
correlation coefficient computed between a criterion and a set of pre- 
dictors, to obtain cross-validation composite correlation coefficients. 
This phase of the experiment was called the cross-validation phase. 


The data obtained in this phase formed the basis for the ensuing 
data analysis? 


Results 


The standard Fisher Transformation (Cramér, 1946), Zr|B = 
(1/2) In (1 + R.|B)/(1 — F.|B), was applied to the composite 

2 Since this experiment may be repli i j 

aei plicated by using the four A matrices to- 
RA ma a Gea ees Program, the generated score data are not avail- 
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TABLE 1 
The Four Factor Structures (A Matrices)* 


Low Homogeneity High Homogeneity 
Factor Factor 
Test 1 2 3 4 Test 1 2 3 4 
1 .30 .20 1 .80 .10 
2 .30 =. 60 2 90 .30 
3 .20 -60 3 .90 40 
4 20 -20 4 .70 10 
5 -40 -30 5 -80 -0- 
6 40 10 6 -70 -20 
@ .30 40 .50 .20 © .30 .40 .50 .20 
Factor Factor 
LE R A TAE ata) PENS A EAEAN TE AN 
Test 1 2 3 4 Test 1 2 3 4 
1 :20 10 1 .80 .20 
2 E U nye tL: 2 .80 .20 
3 10 .80 3 .80  .30 
4 320% 810 4 80 .40 
5 -30 .10 5 -80 -10 
6 -30 .40 6 .80 40 
7 -20 .30 8; .80 -20 
8 .20 -10 8 -80 30 
9 10 10 9 ,80 -0- 
10 30 +380 10 .80 -20 
ll 30 10 11 -80 .40 
12 10 10 12 .80 .55 
(0 .30 .40 .50 .20 > .30 .40 .50 .20 


a The unique factor coefficients have been omitted. They may be calculated for any test, i, 
from the following formula: 
us = (1 — Ejup) 


ò @ refers to the criterion. 


correlation data to produce data suitable for analysis which trans- 
formed data is presented in Table 2 with decimal points omitted 
(decimal points appeared in front of each three digit number). A 
standard fixed-factor, four way ANOVA was then performed on 
the data presented in Table 2. The results of the ANOVA are pre- 
sented in Table 3. 

In order to allow for any possible biases in the F tests due to un- 
equal cell variances (McNemar, 1949), a significance point of .025 
was set, thereby insuring a true significance point not exceeding 
050 in value. The only significant sources of variation were the 
main effect of N and the triple order interaction of NSM. The 
Newman-Keuls procedure for individual comparisons (Winer, 
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TABLE 2 
Cross-validation Data After Fisher Transformation 


M 
Within ee A 
HE 8 ON Cell Wr RA WG+1 WG+I 
1 250 509 385 389 
2 729 233 589 389 
3 2372422 900 333 
50 4 211-710 203 587 
5 504 460 539 090 
z 1931 2334 2616 1788 
; x 386 467 523 358 
1 640 564 390 583 
2 624 469 569 533 
3 450 391 556 559 
200 4 411 585 295 560 
5 478 568 461 439 
z 2603 2577 2271 2674 
x 521 515 454 535 
LO 
1 328 482 480 534 
2 497 653 102 296 
3 395 458 358 455 
50 4 217 333 147 503 
5 459 544 —001 506 
z 1896 2470 1086 2294 
a x 379 = 494 217 459 
1 555 527 377 413 
2 556 528 552 569 
3 665 451 591 462 
ROTIRE NA 493 484 307 477 
5 348 427 496 461 
2X 2617 2417 2323 2382 
eo 


underline joining any two sums in Table 4 indicates that the cor- 
Tesponding means (one-tenth of the sums) did not differ signifi- 
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TABLE 2 (Continued) 


M 
Within —— 
H 8 N Cell WT RA WG+1 WG+I 
1 437 412 562 438 
2 377 665 504 692 
3 167 769 130 452 
50 4 400 484 509 353 
5 431 463 720 647 
zX 1812 2793 2425 2582 
Žž 362 559 485 516 
6 
1 482 631 509 458 
2 596 531 337 494 
3 598 499 459 466 
200 4 713 514 519 561 
5 524 417 455 469 
=x 2913 2592 2279 2448 
x 583 518 456 490 
HI 
1 658 360 357 579 
2 400 400 349 697 
3 219 371 568 417 
50 4 320 372 525 474 
5 468 193 507 833 
2x 2065 1696 2306 3000 
-q 413 339 461 600 
12 
s 444 334 469 481 
2 491 538 389 505 
3 383 419 429 409 
200 4 472 516 528 443 
5 400 497 520 528 
=X 2190 2304 2335 2366 
x 438 461 467 473 


TTS 


Discussion and Conclusions 


There is no need to explain or interpret the significant main ef- 
fect of N since, according to previous research, this was to be ex- 
pected. The significant interaction effect of NXSXM indicated that 
sample size, size of test battery, and method jointly interacted 
among themselves. This joint effect can be more clearly grasped by 
considering what method produced the largest mean at each of the 
four possible treatment combinations of the levels of N and S. The 
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TABLE 3 
ANOVA Source Table 


— 
Source SS DF MS F k 
N 109,093 1 109,093 7.24s 
Ss 52,237 1 52,237 8.47 
H 20,862 1 20,862 1.38 | 
M 61,505 3 20,502 1.36 
NXS 1,012 1 1,012 <1? 
NXE 46,596 1 46,596 3.09 
NXM 87,373 3 29,124 1.93 
SXH 466 1 466 <1 
SXM 70,352 3 23,451 1.56 
HXM 50,589 3 16,863 1.1210 
NXSXH 5,618 1 5,618 <1? 
NXSXM 146 ,347 3 48 ,782 3,240 
NXHXM 56,594 3 18,865 1.25 
SXHXM 101,309 3 33,770 2.24 7 
NXSXHXM 102,885 3 34,295 2.28 1 
Error 1,929,174 128 15,072 = 
Total 2,842,012 159 peie yy 


© Significant at the .025 point. 
» Not significantly Jess than one at .025 point. 


four possible combinations of N and S were 50(12), 50(6), 200(12),. 
and 200(6), and the best methods for these combinations were 
WG + I, RA, WT, and WT, respectively. The unexpected result of © 
RA working well for a small sample size and a small number of © 
variables was difficult to explain except on the basis of sampling er- ~ 
tor, The mean at this treatment combination for WG-1 was very — 
little different from that of RA as can be seen in Figure 1. This 
tended to support the sampling error explanation. The two means j 
were not significantly different from each other according to 
Table 4. 

The order of the four treatment combinations as laid out above 
appeared to be in the same order (descending) of expected overfit, 
holding method constant, Except for the questionable result for 
RA, the ordering of best methods corresponding to the treatment 
combinations as laid out above was about in ascending order of ex- 
pected overfit, in regard to method alone. This result was reasonable ~ 
and indicated that the variable of expected overfit was an impor- 
tant variable in determining what method worked best for a given — 
combination of sample size and size of test battery. Another impor- | 
tant variable seemed to be expected shrinkage which is very closely i 
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Figure 1—N x S x M Interaction 


related to, but not completely determined by, expected overfit 
(overfit involves only one source of sampling error whereas shrink- 
age involves two sources of sampling error). It is probable that the 
operation of these two variables was the chief determiner of what 
method worked best for a given treatment combination. It should 
be noted that the ranking of methods in regard to expected shrink- 
age is, in general, the same as that for expected overfit. 

The fact that no significant interaction effects other than 
NXSXM were found in the present study suggested that an ex- 
treme sample size alone or an extreme size of test battery alone 

. probably did not produce great differences among methods, or, in 
other words, that, for this case, any one of the four test selection 
methods probably would have worked equally well. Since H did not 
appear in any significant effect, it must be concluded that the size 
of predictor incorrelations was probably not an important variable 

-in determining what method to use in a given situation, This is an 
important practical conclusion since it indicates that it is not neces- 
sary to evaluate predictor intercorrelations in determining what 
test selection method to use in a practical situation. 

i A the additional variable of cost or effort of method were taken 
in 

from the significant NXSXM effect. The order (ascending) of the 
four methods under consideration according to cost or effort was 


WG-+L, respectively. Thus it would seem that the Wherry-Gaylord 


Test Selection Method is a far better method than it has been given 


e 


m LESTER C. SHINE II 343 


credit for in the literature. This method also has the distinct ad- 
vantages that it is equally applicable to test selection or item selec- 
tion, and that it is easy to understand and apply. Further, the small- 
est and largest transformed means for the best methods and best 
least costly methods were .466 and .552, respectively. These values 
are equivalent to composite correlations of .435 and .502, respec- 
tively. In practice, such correlation values would frequently not be 
considered to differ enough to warrant extra expense. 

The interpretation of the significant NSM interaction has 
depended heavily upon the assumption that the assumed linear 
nature of the interaction would still hold if medium-valued levels 
had been included for N, S, and H. This assumption may not be 
warranted and should be checked in future research. If H were as- 
sumed to appear in no significant effects, the assumption could be 
inyestigated by allowing N and S to have three levels, say 6, 9, 12 
ahd 50, 100, 200, respectively, with H being held constant at a 
medium level. If a within cell n of 5 were used along with the four 
methods studied in the present investigation, a total of 42,000 
fictitious subjects would be required. Thus the size of this proposed 
experiment would be about the same as that of the present study. 

No negative correlations were allowed in the population inter- 
correlation matrices used in this investigation. The effects of such 
correlations should be investigated in future research. If the assump- 
tion were made that H would not be involved in any significant 
effects, than a suitable investigation could be made by modifying 
the present design such that H is held constant at a medium level 
and that two additional method levels are added to M. The two 
new methods would be the Wherry-Gaylord Test Selection method 
allowing plus or minus unit weights or zero and the Wherry-Gay- 
lord Test Selection method allowing plus or minus integer weights 
or zero. The population intercorrelation matrices should, of course, 
have both negative and positive correlations of such a nature that 
reversing test scoring would not remove the differential sign pat- 
` tern. If a within cell n of 5 were used, a total of 30,000 fictitious 
subjects would be required for the proposed design. 


Summary 


The present investigation dealt with the relative efficiency of four 
representative test selection methods in cross-validation, using com- 
puter generated data. The four methods were chosen on the basis of 


p 
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previous research and were the Wherry Test Selection method, 
Regression Analysis, the Wherry-Gaylord Test Selection method al- 
lowing plus unit weights or zero only, and the Wherry-Gaylord 
Test Selection method allowing plus integer weights or zero only, 
denoted by WT, RA, WG-++1, and WG-LI, respectively. The effi- 
ciency of the four methods was studied under manipulation of the 
variables of sample size, number of predictors in the test battery, 
and homogeneity of the test battery (measured by size of predictor 
intercorrelations), denoted by N, S, and H, respectively, with 
method being denoted by M. A fixed factor, four-way ANOVA 
with no repeated measures and a within cell n of 5 was proposed to 
analyze the data. The levels of N, S, and H were 50 and 200, 6 
and 12, and Low and High, respectively. The levels of M were the 
four chosen test selection methods. 

The four methods were applied in validation to randomly drawn 
intercorrelation matrices with built-in appropriate N, S, and H 
characteristics. The resulting test selection weights were then ap- 
plied in cross-validation to randomly drawn appropriate intercor- 
relation matrices to obtain cross-validation composite correlation 
coefficients. These coefficients were transformed, by using the stand- 
ard Fisher Transformation, to insure normality of data. The pro- 
posed 2X2X2Xx4 ANOVA was then applied to the transformed 
data, with a significance point of .025 being set in advance. The 
only significant effects found were the N main effect, which was ex- 
pected, and the N XSXM interaction. 

The best methods for the four N, S treatment combinations of 
50(12), 50 (6), 200 (12), and 200 (6) were WGI, RA, WT, and WT, 
Tespectively, The result for 50(6) was questionable on a sampling 
error basis, and it was noted that for this particular treatment 
combination, WG+1 was only slightly less efficient than RA. It was 
concluded that the joint operation of the effects of expected overfit 
and expected shrinkage probably determined what test selection 
method works best for a given treatment combination. The fact that 
H appeared in no significant effect indicated that size of test inter- 
correlations was not an important variable to take into account 
in making a decision as to what test selection method to use in 
4 given practical situation, 

If the additional variable of cost or effort of method is taken into 
account, the best methods for the 50(12), 50(6), 200(12), and 
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200(6) treatment combinations of N and S were WGI, WG-++1, 
WG-41, and WG-+I, respectively. It was concluded that the 
Wherry-Gaylord Test Selection method, with its advantages of be- 
ing equally applicable to test or item selection and of being easy to 
understand and apply, is a far better test selection method than 
it has been given credit for in the literature. 

Since the interpretation of the significant NSM interaction 
depended heavily upon the assumption that its assumed linear na- 
ture would not change if medium-valued levels had been included 
for N and S, an experiment was proposed to investigate the validity 
of the assumption. Further, since no negative correlations were al- 
lowed in the population intercorrelation matrices from which sample 
matrices were drawn, an experiment was proposed to investigate the 
effects of such correlations upon the efficiency of test selection 
methods. 
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A RESEARCH NOTE ON THE METHOD OF ERROR-CHOICE 


WARREN S. BLUMENFELD+ 
Purdue University 


Tur method of error-choice has been described by Remmers 
(1954, p. 239), and it has been applied by Bernberg (1951), Ham- 
mond (1948), Kubany (1953), and Wechsler (1950a; 1950b). The 
method involves providing the subject with an unstructured, some- 
what ambiguous, situation (question) in which his choice of item 
alternatives reveals his attitude. Operationally, this entails pro- 
viding alternatives equi-distant from the true answer; but not 
providing a correct answer. The direction of deviation from the non- 
appearing correct alternative indicates the respondent’s attitude. 


Purpose 


The purpose of this investigation was to determine the extent of 
relationship between an error-choice item and a criterion of affect 
for an attitude object. 


Procedure 


Data Collection 


The data for this investigation were collected through the opera- 
tions of the Purdue Opinion Panel (Gage and Remmers, 1948), A 
questionnaire, the topic of which was attitudes toward physicians, 
‘was administered nationally to high school students and to one 
parent of each student; it contained three item types—biographic, 


1 This report was adapted from a section of a doctoral dissertation submitted 
in partial fulfillment of the requirements of the Ph.D. degree at Purdue Uni- 
yersity, 1963. The author is now a Research Psychologist with the National 
Merit Scholarship Corporation at Evanston, Illinois. 
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semantic differential, and attitudinal. The research samples upon ` 


which this report is based contained 1000 students and one parent 
of each of the students, i.e., 1000 students and 1000 adults. The stu- 
dent sample was representative of the national population of high 
school students; the parent sample was defined by the student sam- 
ple (Blumenfeld, Franklin, and Remmers, 1963). 


Criterion 


From the responses to the semantic differential scales, a com- 
posite image criterion for students and a composite image criterion 
for parents was developed which reflected individual differences in 
affect for the attitude object, physician. The interrater reliability 
of the student criterion was .82; the interrater reliability of the 
parent criterion was .83. The criterion development is described in 
detail by Blumenfeld ( 1963). 


Error-Choice Item 


The error-choice item administered to both students and parents 
was “How many years are there between the time a physician first 
enters college and the time he completes internship? The typical in- 
terim time, 9 years, did not appear; item alternatives provided were 
6, 8, 10, and 12 years. The rationale of the error-choice item would 
indicate that selection of 6 or 8 connotes negative affect and that 
the selection of 10 or 12 connotes positive affect. That is, if the re- 
spondent holds physicians (the attitude object) in high esteem, he 
will attribute more academic preparation to physicians than is ob- 
jectively the case; the more the Positive affect, the more the aca- 
demic preparation attributed, 


Data Analysis 


To determine the relationship of the error- 
criterion, the hypothesis of no difference be 
terion, scores associated with 


tested by means of analysis of variance (Winer, 1962, p. 96). In 
addition, to determine the relation: 


c ship of the error-choice item with 
other items in the questionnaire, the item was tested by being paired 
with several other items which were judged to be pertinent. The 
hypothesis of zero interaction between item response distributions 
was tested by means of chi Square (Siegel, 1956, p. 104). 


choice item with the 
tween the mean cri- 
item alternative selection was 
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The analyses of variance in the student and parent samples 
yielded F statistics of 2.00 (df = 3,996) and 1.50 (df = 3,990), 
respectively. Neither of these obtained statistics is significant at the 
.05 level. That is, in both samples, the differences between the 
mean criterion scores associated with the item alternative selection 
could have arisen by chance. 

Tables 1 and 2 summarize the chi square tests of interaction be- 
tween the error-choice item and other items judged to be pertinent 
for the student and parent samples, respectively, Relationships sig- 
nificant at or beyond the .05 level are identified. 


Discussion and Conclusion 


Inspection of those items significantly related to the error-choice 
item in the student sample indicated that the higher the grade level 
in school, the longer the physician’s educational preparation was 
thought to be; boys ascribed less time than did girls; white stu- 
dents ascribed more time to academic preparation than did others; 
better students perceived the process as longer than did poorer 
students; students who said they would want to become a physician 


TABLE 1 


Summary 1i Results of Chi Square Tests of the Hypothesis of Zero Interaction 
etween Student Perception of Physician's Education 
and Selected Pertinent Variables 


Variable df Chi Square 

Grade in school 6 17.57** 
Sex of student 3 45.72** 
Socio-economic status 6 5.52 
Political preference of student 6 3.22 
Color 3 14.40** 
Curriculum 9 16.45 
Grades 9 19.64** 
Religion 9 10.32 
Desire of student to become physician 9 33.87** 
Physician in family 3 0.29 
Ben Casey viewing 9 444 .09** 
Dr. Kildare viewing 9 836 .48** 
Number of visits to physician in 

past year 9 14.08 
Parent AA of physician’s 

education 9 5.83 


* Significant at .05 level. 
** Significant at .01 level. 
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TABLE 2 
Summary of Results of Chi Square Tests of the Hypothesis of Zero Interaction 
between Parent Perception of Physician’s Education 
and Selected Pertinent Variables 


Variable df Chi Square 

Family breadwinner 3 2.28 
Sex of responding parent 3 6.63 
Physician 3 0.38 
Age of parent 12 7.71 
Education of parent 12 35.18** 
Occupation 12 17.22 
Political preference of parent 6 23.33** 
Number of family visits to physician 

in past year 12 15.87 
Desire for child to become physician 9 7.96 
Ben Casey viewing 9 24,74"* 
Dr. Kildare viewing 9 19.75* 
Attitude toward physician’s fees 6 8.87 
Attitude toward Medicare 9 9.12 


* Significant at .05 level. 

** Significant at .01 level. 
saw the educational process as being shorter than did the students 
who were not interested; and those who viewed TV medic dramas 
more frequently felt that the educational process was shorter than 
did the less avid TV medic viewers, 

For those items significantly related to the error-choice item in the 
parent sample, inspection indicated that the more education a par- 
ent had, the longer was the perception of education for physicians; 
Democrats tended to under-estimate the time while Republicans 
tended to over-estimate; and, as with the students, the more steady 
TV medic viewers tended to under-estimate the academic prepara- 
tion while the less steady viewers tended to over-estimate. 

In conclusion, the data in this investigation indicated that the 
error-choice item was not related to the criterion of attitude ob- 
ject affect in either the student sample or the parent sample; how- 
ever, the item was related to student grade in school, sex of student, 
color of student, grades attained, desire of the student to become & 
physician, student viewing of TV medics, education of parent, po- 
litical preference of parent, and parent viewing of TV medics. 
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COMPARISON OF PERSONALITY AND ATTITUDE 
VARIABLES 


ANDREW L. COMREY 
University of California, Los Angeles 


Apprication of the factored homogeneous item dimension ap- 
proach (Comrey, 1961) in a series of investigations (Comrey, 1962, 
1964, 1965; Comrey and Schlesinger, 1962) has led to the develop- 
ment of a system of factors intended to describe certain aspects of 
the human personality. The same type of approach applied in the 
area of political and social attitudes resulted in the definition of 
several substantially correlated primary level factors which in 
turn produced a second-order general factor of Radicalism-Con- 
servatism (Comrey and Newmeyer, 1965). The present investigation 
was undertaken for the purpose of determining the interrelation- 
ship, if any, of these two domains and in particular to ascertain 
whether any of the attitude measures might prove useful in measur- 
ing the main personality factors. 

A 216 item inventory was prepared which consisted of 51 fac- 
tored homogeneous item dimensions, or FHIDs (Comrey, 1961), a 
social desirability scale of eight items, and a four item validity 
scale designed to detect random marking. Each FHID consisted 
of four items which were written to measure a specifically conceived 
variable and hence to correlate substantially with each other. Each 
FHID, except one, was included in the analysis because it had a 
substantial loading on one of the personality or attitude factors 
in at least one previous analysis, or it was newly conceived for this 
investigation with the hope that it would be highly related to one of 
these factors. 

Each FHID brought over from a previous study had emerged as 
an item factor in a factor analysis of items. Only items with load- 


853 


854 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


ings of approximately .45 or more were retained for inclusion 
within the FHID. Thus, items in a given FHID belong there by 
the joint criterion of logical and statistical affiliation. The per- 
sonality factors of major importance were determined from correla- 
tions among FHIDs rather than from correlations among items. 
That is, each personality and attitude factor in this study was de- 
fined by highly loaded FHIDs rather than single items. This tech- 
nique has the effect of providing a greater degree of stability to the 
structure of important factors than is normally likely to occur with 
analysis of single items. Furthermore, appearance of item specific 
factors tends to be restricted to the FHID level rather than creep- 
ing into the structure of the major factors. In using single items to 
search for the important factors, on the other hand, it is easy to 
produce factors which are more like FHIDs than important per- 
sonality factors, since their most heavily loaded variables consist 
of very similar kinds of items. 

Personality factors included in this study which were defined by 
four or more FHIDs were: Empathy, Neuroticism, Shyness, Com- 
pulsion, Dependence, and Hostility. Ascendance and Self Control 
were two additional personality factors defined respectively by 
three and two FHIDs each. Attitude factors included were: Wel- 
fare State Attitudes, Religious Attitudes, Punitive Attitudes, Na- 
tionalism, and Racial Tolerance Attitudes. Either two or three 
FHIDs were included to measure each of these attitude factors. 

The items for a given FHID were well separated in the inventory 
booklet. A subject’s response to each item was selected from one of 
two 9-choice scales and recorded on a separate answer sheet. The 
first response scale ranged from “1, Never” to “9. Always.” The 
second ranged from “1. Absolutely Not” to “9. Absolutely.” Vol- 
unteer university students and community persons (N = 446) 
served as subjects with a promise of an individual analysis of their 
test results as an inducement to participate. Community persons 
Were obtained by requests for participation distributed in ran- 
domly selected blocks within a ten mile radius of UCLA. Subjects 
were about equally divided as to sex and the student-non-student 
a Ce e was between 26 and 27. The average edu- 
scale of political same te the a bake eee aay 
slightly on the demoeratie a group showed a central tendency 
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A total score over each FHID for each subject was computed by 
adding up the numerical responses selected. These total scores were 
intercorrelated to provide a basis for dividing up the FHIDs into 
four item analysis groups. Highly correlated FHIDs were placed in 
different item analysis groups. Items from all the FHIDs in- 
cluded in a given item analysis group were then intercorrelated by 
Pearson r and factor analyzed for item analysis purposes. This was 
done to confirm previously developed FHIDs as item factors in 
this study and to check on new ones. If a FHID failed to emerge 
as an item factor in one of these analyses, it was dropped. Any item 
within a given FHID which failed to have an item factor loading 
of about .45 or more was dropped. Eight FHIDs lost one or two 
items out of four in this way. Correlations among FHIDs were 
corrected for attenuation in both variables to determine if any 
such value approached 1.00. This occurred for the FHIDs Racial 
Tolerance and Rapid Social Change, two FHIDs which defined an 
attitude factor in the previous study of attitudes. The five best 
items from these two FHIDs were combined into one FHID, named 
“Racial Tolerance.” This was done to prevent the emergence of a 
factor in the analysis of FHIDs which would be merely a doublet 
based on these two alternate forms of the same FHID. 

Total scores over the retained items in the 50 remaining refined 
FHIDs were calculated. These variables together with total scores 
over eight social desirability items and scores on four background 
variables provided the following list for the main analysis of 
FHIDs: 

1. Acceptance of Authority, 2. Lack of Ego Strength, 3. Love of 
Routine, 4. Censorship, 5. Agreeableness, 6. Rhathymia, 7. Shy- 
ness, 8. Pacifism, 9. Status Quo, 10. Agitation, 11. Ascendance, 12. 
Racial Tolerance, 13. Cautiousness, 14. Service Orientation, 15. 
Reaction to Authority, 16. Weak Federal Government, 17. Need 
for Approval, 18. Talkativeness, 19. Depression, 20. Capital Pun- 
ishment, 21. Defensiveness, 22. Order, 23. Cynicism, 24. World Gov- 
ernment, 25. Self Sufficiency, 26. Desire for Power, 27. Social De- 
sirability, 28. Religiosity, 29. Inferiority Feelings, 30. Need to 
Excel, 31. Personal Grooming, 32. Succorance, 33. Self Control, 34. 
Hostility, 35. Submission, 36. Friendliness, 37. Adequacy, 38. Con- 
formity, 39. Welfarism, 40. Meticulousness, 41. Achievement, 42. 
Helpfulness, 43. Severe Treatment of Criminals, 44. Dependence, 
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45. Psychopathic Deviation, 46. Stage Fright, 47. National Serv- 
ice, 48. Pessimism, 49. Sympathy, 50. Drive to Finish, 51. Con- 
traception, 52. Age, 53. Sex (1 male, 0 female), 54. Educational 
level, and 55. Political Preference (higher scores for liberals, lower 
scores for conservatives). 

The 55 X 55 matrix of Pearson intercorrelations among these 
variables was factor analyzed by the minimum residual method, 
which requires no communality estimates since it operates on the 
off-diagonal elements only (Comrey and Ahumada, 1964). The 
minimum residual method automatically terminates extraction 
of factors when the largest remaining factor behaves as though it 
were associated with a negative eigen value. In this study, 19 fac- 
tors were extracted before this occurred. Four of these factors had no 
loading greater than .3 and one additional factor had only one 
such loading, namely, .33. The factor solution was iterated with 14 
factors, therefore, each time reinserting the communalities cumu- 
lated from the previous iteration. The communalities were reason- 
ably stable after 10 iterations. The second and succeeding solu- 
tions were principal factor solutions, since the minimum residual 
method becomes a principal factor method when the diagonal cells 
are employed in the calculations. 

The 14 stabilized unrotated factors were rotated analytically by 
the normal varimax method (Kaiser, 1958) and further rotated 
visually to oblique simple structure, The orthogonal solution is so 
close to the oblique solution, however, that only the orthogonal 
solution tables will be presented. Loadings of .8 or more on these 
14 orthogonal rotated factors were as follows: 

Empathy. Helpfulness, .76; Service Orientation, .69; Sympa- 
thy, .68; Social Desirability, 46; Dependence, .35; and Friendli- 
ness, .34. 
ts ve aoe Depression, 78; Lack of Ego Strength, .73; Pes- 

» 71; Agitation, .68; Adequacy, —.65; Inferiority Feelings, 
60; Shyness, 36; Submission, 34; Hostility, 30. 
Route Mian: Need for Order, 76; Drive to Finish, .62; Love of 
Pak Acta eticulousness, .56; Personal Grooming, .43; and 

autiousness, 41, 
nit hry Sam eh; Danae, — 

»—-40; Submission, .40; Friendliness ,—.40, 
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Hostility. Cynicism, .65; Hostility, 59; Psychopathic Devia- 
tion, 55; Rhathymia, .48; and Defensiveness, .39. 

Dependence. Conformity, .70; Need for Approval, .68; Succor- 
ance, 46; Self Sufficiency, —.41; and Submission, .32. 

Achievement Need. Achievement, .82; Need to Excel, .64; Desire 
for Power, .63; and Adequacy, .31. 

Self Control. Self Control, .66; Agreeableness, .63. 

Ascendance. Ascendance, .71; Submission, —.43; and Defensive- 
ness, .33. 

Welfare State Attitudes. Welfarism, .72; Weak Federal Gov- 
ernment, —.72; Racial Tolerance, .66; World Government, .61; Po- 
litical Preference, .55; Pacifism, .41; and Severe Treatment of 
Criminals, —.31. 

Religious Attitudes. Censorship, .72; Contraception, —.70; Re- 
ligiosity, .65; National Service, .82; Severe Treatment of Crimi- 
nals, 31; and Rhathymia, —.30. 

Punitive Attitudes. Capital Punishment, .54; Pacifism, —.50; 
National Service, .45; Severe Treatment of Criminals, 39; and 
Defensiveness, .35. 

Sex, Personal Grooming, .51; Status Quo, .44; Educational Level, 
—.43; Sex, —.42; Reaction to Authority, —.42; National Service, 
.38; Social Desirability, .35; and Acceptance of Authority, .33. 

Age, Age, .70. 

Empathy, Neuroticism, Compulsion, Shyness, Hostility, and De- 
pendence have been identified as major personality factors in sev- 
eral previous studies. Achievement Need emerged here for the first 
time in this series of investigations due to the addition in this 
study of the Achievement FHID to the list of variables. Self Con- 
trol and Ascendance have appeared in previous studies but little 
success has been experienced to date in finding additional FHIDs 
to measure these factors. 

Welfare State Attitudes, Religious Attitudes, and Punitive At- 
titudes each appeared as a factor in the previous study of political 
and social attitudes. The Racial Tolerance attitude factor from 
that study was not found here because the two FHIDs defining it 
were combined into one variable in the present study. The factor 
of Nationalism in that study, defined by National Service, Paci- 
fism, and World Government failed to achieve separate definition 
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in the current analysis, but rather split up and combined with Wel- 
fare State Attitudes and Punitive Attitudes. There is a strong 
second order factor running through the attitude factors in both 
studies, however, and the present sample had far fewer politically 
conservative subjects. These facts may have been responsible for 
the disappearance of Nationalism as a separate factor here. 

The major personality factors expected to emerge from this 
investigation did so in clear fashion, Only one of the major attitude 
primaries which could have been expected to emerge failed to do 
so. Two personality variables had small loadings on attitude fac- 
tors, but no attitude variable had a loading as high as .3 on any 
personality factor. It would appear, therefore, that there is little 
hope of broadening the measurement base for these personality 
factors by using attitude variables of this kind. 

Although an orthogonal solution was used, it is clear that cor- 
relations do exist among these factors, particularly the attitude 
factors. To study these interrelationships, factor scores were ob- 
tained for all factors except Age and Sex by adding item scores 
over those items belonging to FHIDs which had loadings of .39 or 
more on the factor. Exceptions were the omission of Social Desir- 
ability from Empathy, Submission from Shyness, and Pacifism 
from Welfare State Attitudes. Social Desirability was retained as 
a separate “factor” score while the other two FHIDs had higher 
loadings on other factors. In the case where a FHID had a nega- 
tive loading on a factor, its item scores were reversed by subtrac- 
tion from 10 before adding them into the factor score. Means, 
standard deviations, and split-half reliabilities corrected by the 


Spearman-Brown formula are given for these factor scores in the 
ADI materials.t 


ma AEE 
1The computations for this stud: i p 
ates te s y were carried out on the IBM 7094 op: 
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The 12 factor scores and the Social Desirability scores were in- 
tercorrelated and factor analyzed by the minimum residual 
method. Only three factors could be extracted. These were rotated 
by the normal varimax method. The first had rotated loadings as 
follows: Punitive Attitudes, —.83; Welfare State Attitudes, .56; 
Religious Attitudes, —.52; and Compulsion, —.32. It is clearly the 
second-order factor of Radicalism-Conservatism also found in the 
first attitude study. The second factor had the following rotated 
loadings: Social Desirability, .77; Empathy, .70; Hostility, 
—.52; and Compulsion, .50, It can be referred to as “Social De- 
sirability” without settling the question whether faking or sheer 
“goodness” is the main content of the factor. Probably some com- 
plex composite of the two is involved. The third second-order fac- 
tor had the following loadings: Neuroticism, .74; Shyness, .56; 
Ascendance, .43; and Hostility, .31. This factor might be classified 
as Poor Social Adjustment. 

The personality factors Empathy, Neuroticism, Hostility, Com- 
pulsion, Shyness, and Dependence have been repeatedly confirmed 
in this series of investigations. Welfare State Attitudes, Religious 
Attitudes, and Punitive Attitudes, and the general second-order 
factor of Radicalism-Conservatism defined by them, have been 
obtained twice. The scales measuring these particular factors, 
which are provided in the ADI materials, all have split-half re- 
liabilities of .87 or more. Such scales should be useful in theoret- 
ical and applied research studies where personality or political 
and social attitude questionnaire measures are needed. 
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A FACTOR ANALYTIC STUDY OF THE SELF-RATINGS 
OF COLLEGE FRESHMEN 


JAMES M. RICHARDS, JR+ 
American College Testing Program 


Ir a non-psychologist wished to learn something about another 
person, he would be very likely to ask a direct question, and he 
would frequently find out what he wanted to know. Psychologists, 
however, have neglected direct self-reports as a source of informa- 
tion about individuals, and have preferred test scores and ratings 
by teachers, supervisors, psychologists, ete. The reason for psychol- 
ogists’ infrequent use of direct self-reports is uncertain, but it is 
probably due to the ease with which self-reports can be dis- 
torted and because of a rather patronizing view of man’s ra- 
tionality, honesty, and self-insight. 

There is, however, a growing body of evidence that suggests 
that the neglect of direct self-reports has undully handicapped 
psychologists in their attempt to understand and predict behav- 
ior. For example, in one study, self-ratings were among the best 
predictors of the on-the-job performance of physical scientists 
(Taylor, Smith, Ghiselin, and Ellison, 1961), and in studies of 
high ability college students, self-ratings have been shown to con- 
tribute significantly to the prediction of high-level achievement in 
culturally important areas of behavior (Holland, 1963; Holland 
and Nichols, 1964). Similarly, Campbell (1959) has proposed that 
an important indicator of test validity should be that “test scores 
predict independent trait-appropriate or criterion measures better 
than do self-ratings” and pointed out that the available evidence 
(Campbell and Fiske, 1959) suggests that only rare tests can be 
considered valid by this standard. 

1 The author would like to thank John Holland, Sandra Lutz, and Clifford 
Abe for their many contributions to this project. 
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To use self-ratings effectively in research or practice, however, 
it is desirable to organize them into a relatively small number of 
categories. The present study, consequently, was conducted to 
develop a clear and sound basis for organizing self-ratings by col- 
lege freshmen on a number of common traits into a brief profile. 
This brief profile can then be used in subsequent research to study 
more efficiently problems in which self-ratings provide useful infor- 
mation. The basic technique was to factor analyze 31 self-ratings 
for a broad cross-section of American college freshmen in di- 
verse colleges. The present study parallels the author’s earlier 
factor analysis of the life goals of college freshmen (Richards, 
1965). It was believed that more useful and interpretable results 
would be obtained by separate factor analyses of life goals and 
self-ratings rather than by a single factor analysis of goals and 
tatings like that done by Astin and Nichols (1964). 


Method 


The present study grew out of the American College Survey 
(Abe, Holland, Lutz, and Richards, 1965) which was conducted 
by the American College Testing Program to obtain a more com- 
plete account of the typical American college student and the vari- 
ation among students from college to college. To accomplish this 
task, a comprehensive assessment was administered in the months 
of April or May, 1964, to 12,432 college freshmen in 31 institu- 
tions of higher education including liberal arts colleges, state uni- 
versities, teachers colleges, and two-year community colleges, This 
group of freshmen, of whom 6289 were men and 6143 were 
women,, provided the sample for this study. 

In this study sample, 7 per cent were enrolled in junior colleges, 
12 per cent in four-year undergraduate colleges, and 81 per cent 
1 universities offering graduate work. Approximately 15 per cent 
vane students in private colleges, while 85 per cent were stu- 
ies in Public colleges, About 95 per cent attended coeducational 
a MAE AAT se were enrolled in colleges in the 

eges in the South, 20 per cent in col- 


R E: but only 3 per cent in colleges on the West Coast. 
e these figures it would appear that students in coeducational 
comeges are somewhat over-represented and students in West Coast 
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colleges are considerably under-represented in the sample. Never- 
theless, the over-all characteristics of the sample suggest that it 
represents a reasonable cross-section of American college fresh- 
men in 1964. 

Scores on a nationally administered test of academic potential 
(the ACT test battery) were available for 7262 of these fresh- 
men. A comparison was made between the distribution of test 
scores in this sub-sample and the corresponding distribution in a 
national norm group (Holland and Richards, 1965). The results 
revealed that on all ACT subtests the sample includes fewer 
persons with low scores than does the national norm group. These 
differences probably occurred because the norm group consisted 
of potentially college-bound high school seniors while the sample 
consisted of college freshmen who had already survived more than 
one half of the academic year. However, a full range of talent is 
represented in the sample, and it differs only slightly from the na- 
tional distribution. 

The assessment device used to collect data was the American 
College Survey (Abe et al., 1965), a booklet which contains a 
letter explaining the purpose of the survey and a series of sections 
planned to elicit information about each student’s achievement, as- 
pirations, attitudes, interests, potentials, values, and background. 
The American College Survey was administered at each college by 
personnel at that college. The survey was filled out by students— 
who recorded their 1004 responses on two special answer sheets— 
in English classes, chapels, and conyocations or in dormitories and 
their homes. 

For the present study, thirty-one self-ratings on common 
traits were used. Each of the subjects rated himself on each of the 
thirty-one traits on a four-point scale (“Below Average,” “Aver- 
age,” “Above Average,” “Top Ten Per Cent”), and scores from 
one to four were assigned to these responses so that a higher score 
indicated a greater possession of the trait in question. A complete 
list of these traits, together with the means and standard devia- 
tions for each sex is shown in Table 1. 

Product moment correlations between the self-rating items were 
computed separately for each sex.* The two resulting 31 X 31 


2 Calculations were carried out at the Measurement Research Center, Uni- 
versity of Iowa and at the University of Utah Computer Center. 
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matrices were factor analyzed using the principal components 
method based on eigenvalues and eigenvectors with unity in the 
diagonal and extracting all factors with an eigenvalue greater than 
1.00. This procedure, including the use of unity in the diagonal, is 
Harris’s (1964) Model A factor analysis, and follows the rationale 
presented by Kaiser (1960). An orthogonal rotated solution was com- 
puted by the Varimax procedure (Kaiser, 1958), and an oblique 
rotated solution by the Promax procedure? (Hendrickson and 
White, 1964) with k = 4. 


Results 


For each sex, seven factors had an eigenvalue greater than 1.00 
and were included in the rotations. For both sexes, the total .10 
hyperplane count is substantially higher for the oblique solu- 
tion than for the orthogonal solution. This indicates clearly that 
the oblique solutions should be used in the further interpretation 
of results. Accordingly, the oblique rotated factors for both sexes 
obtained by the Promax procedure (Hendrickson and White, 1964) 
are shown in Table 2, and the correlations among Promax fac- 
tors in Table 3. 

The Coefficient of Congruence (Tucker, 1951) was computed 
between each Promax factor for males and each Promax factor for 
females to determine the extent to which the structure of self-rat- 
ings is similar for men and women. These coefficients are pre- 
sented in Table 4 with female factors rearranged to place highest 
Coefficients of Congruence in the diagonal. 


Discussion 
The results shown in Table 4 indicate a high degree of simi- 
larity between the structure of self-ratings for males and the cor- 


responding structure for females, since a good match is obtained 
for all seven factors for both sexes. Moreover, the match has both 


pe SS 

è Tables showing for each sex the correlations among the self-ratin s, the 
heieri Orgon Totated factors, and the EETEHI matrices for com- 
puting the Promax solutions from the Varimax solutions have been deposited 
as Document number 9040 with the ADI Auxiliary Publications Project, 
Photoduplication Service, Library of Congress, Washington 25, D. C. A copy 
may be secured by citing the Document number and by remitting $1.25 for 
photoprints, or $1.25 for 35 mm. microfilm. Advance payment is required. 
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JAMES M. RICHARDS, JR. 865 


TABLE 1 
Means and Standard Deviations of Self-Ratings for Both Sexes 


Males Females 

Self-Rating Mean §.D. Mean S.D 

1. Originality 2.44 .74 2.34  .70 
2. Leadership 2.43 .79 2.28 .78 
3. Mechanical Ability 2.24 92 1.64 .75 
4, Popularity 2.44 .70 2.35 64 
5. Athletic Ability 2.48 .89 2.15 81 
6. Understanding of Others 2.73  .73 2.82 .69 
7. Drive to Achieve 2.62 .80 260! 807. 
8. Mathematical Ability 2.33 .94 1.97 .89 
9. Scholarship 2.35: 1.78 2.39 .78 
10. Sociability 2.41 .74 Deol A 
11. Artistic Ability 1.74 .85 1.84 .85 
12. Aggressiveness 2.12; ` 14 2.07 .70 
13. Speaking Ability 2.23 .78 217) 78 
14. Self-Control 2.61 .78 2.51 .70 
15. Independence 2.81 .77 2.68 .76 
16. Scientific Ability 2.14 .85 1.73.75 
17. Conservatism 2.26 .74 2.18 .65 
18. Practical Mindedness 2.52.71 2.51 .69 
19. Writing Ability 2.18.77 2.19 .78 
20. Expressiveness 2.24 .71 2.26 .71 
21. Cheerfulness 2.638 .72 2.60 -70 
22. Self-Confidence (Social) 2.27 8L IS ir AREA À 
23. Self-Confidence (Intellectual) 2.34 74 2.18 70 
24, Perseverance 2.87: 67 2.36 .64 
25. Popularity with the opposite sex 2.34.73 2.26 .70 
26. Research Ability 2.436 7,69) 1.91 .64 
27. Physical Energy 2.65 .76 2.38 .71 
28. Sense of Humor 2.76 .72 2.66 .67 
29. Physical Health A Fh Aapa “kd 

5 ‘in; ili . ' i . 

ao ee 2.55 73 a78 Ti 


o 
pa 


. Sensitivity to the needs of others x 


alidity (Campbell and Fiske, 1959). 
The Coefficients of Congruence between matching factors range 
from .90 to .99 with a median of .94, while the coefficients for un- 
matched factors ranged from —.12 to .33 with a median of .01. It 
should be emphasized that the factor analyses and rotations were 
completely independent, and that the samples were large and di- 
verse. These results, therefore, are impressive evidence for a con- 
sistent organization of self-ratings, or of self-concepts, for both 
sexes, Such consistency from sample to sample is an important con- 
sideration in determining the adequacy of representation of the 
domain by the rotated factor solution (Harman, 1960). 


convergent and discriminant v. 
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TABLE 3 
Correlations among Promaz Oblique Factors 


A B Cc D E F G 

A = 24 26 39 -11 51 

B 27 — 39 46 -12 26 i 
Cc 44 21 — 52 —24 45 —04 
D 31 39 30 — —25 46 13 
E 46 33 22 33 — —23 19 
F —03 09 08 —07 —01 a —02 
G 38 19 32 29 25 —01 — 


Note—Correlations for males are shown above the diagonal and for females below. Factors 
are reflected as appropriate. 


TABLE 4 
Coefficients of Congruence between Male and Female Promax Factors 


Males 


a Reflected factor. 


The Promax factors are briefly described and interpreted below: 

Male A-Female C has high loadings for both sexes on the traits: 
“athletic ability,” “physical energy,” and “physical health” with 
a secondary loading on “sense of humor.” An appropriate title 
would be Physical Well Being. 

Male B-Female B has high loadings on the traits: “mathemati- 
cal ability,” “scholarship,” “scientific ability,” and “self-confi- 
dence (intellectual) ,” with secondary loadings on “drive to achieve” 
and “research ability.” This pattern could be summed up by the 
title Scholarship. 

Male C-Female E has high loadings on “originality,” “artistic 
ability,” “speaking ability,” “writing ability,” “expressiveness,” 
and “acting ability.” An obvious title would be Hstheticism, and 
his factor to be correlated with in- 


one would expect scores on t 
dependent estimates of originality. This pattern of loadings im- 
plies that college freshmen conceive of creativity largely in terms 
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of the arts, and have little conception of the role of originality in 
the sciences or other non-artistic fields. 

Male“D-Female D has high loadings on “self-control,” “inde- 
pendence,” “conservatism,” “practical mindedness,” and “perse- 
verence” for both sexes. In addition, it has moderate loadings for 
males, but not for females on the traits “understanding of others” 
and “drive to achieve.” This cluster of traits suggests a well- 
integrated, self-reliant, and pragmatic approach to life. This fac- 
tor thus appears to tap some of the same psychological characteris- 
ties as the “Achievement via Independence” Scale of the CPI 
(Gough, 1957). Perhaps an appropriate title for this factor would 
be Pragmatism. 

Male E-Female F has high loadings on “mechanical ability,” 
“artistic ability,” “scientific ability,” and “research ability.” An 
appropriate title might be Technical-Scientific Ability. The high 
loading for “artistic ability” might indicate that this factor taps 
scientific creativity. 

Male F-Female A has high loadings for both sexes on the traits 
“leadership,” “popularity with opposite sex,” “sociability,” “ag- 
gressiveness” (with a substantially higher loading for females 
than for males), “cheerfulness,” “self-confidence (social),” and 
“popularity.” The overall impression created by these loadings is 
of a pleasant, confident extrovert. The most appropriate title might 
be Sociability. 

Male G-Female G has high loadings on “anderstanding of 
tieni “sensitivity to the needs of others,” “cheerfulness,” and 

sense of humor,” with secondary loadings on “aggressiveness,” 
and “self-control.” This configuration of traits is close to what is 


called “empathy,” and an appropriate title might be Sensitivity 
to Others. 


-ratings were not considered, the vari- 
ting success might be merel; ue 
; : : ly the vari 
ables which correlated with original self-concepts. In such a case, 


| 
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it would be better to use self-ratings directly rather than other 
variables which indirectly reflect the same characteristics. 


Summary and Conclusions 


The primary purpose of this study was to provide a brief pro- 
file of self-ratings which would adequately describe some common 
characteristics of the self-conceptions of American college fresh- 
men. In samples of 6289 male and 6143 female college freshmen, 
31 self-ratings on common traits were intercorreleted. With unity 
in the diagonal, a principal components analysis was carried 
out, extracting all factors with an eigenvalue greater than 1.00. 
An orthogonal rotated solution was computed by the Varimax 
procedure, and an oblique solution by the Promax procedure. The 
oblique solutions were more satisfactory, and were used in assess- 
ing the similarity of the two rotated matrices. Seven factors com- 
mon to both sexes were titled: Physical Well Being, Scholarship, 
Estheticism, Pragmatism, Technical-Scientifie Ability, Sociabil- 
ity, and Sensitivity to Others. No factors unique to one sex were 
obtained. 

The study appears successful in attaining its main purpose, since 
the number of variables was reduced from 31 to 7 for each sex. 
There is a high degree of congruence between the factors for males 
and females; the obtained factors are easily interpreted; and 
the use of large diverse samples lends strong support to our con- 
fidence in the factor pattern. The reduction of self-ratings to 
seven factors provides a simple, brief set of items for assessing self- 


concepts in questionnaires and similar assessment devices so that 
more expensive and time-consuming devices are not always neces- 


sary or desirable. 
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JUDGING COMPLEX VALUE STIMULI: 
AN EXAMINATION AND REVISION OF MORRIS'S 
PATHS OF LIFE* 


PAUL DEMPSEY 
Institute of Behavior Assessment, Davis, California 
AND 
WILLIAM F. DUKES 
University of California, Davis 


Dunne the past few decades the philosopher Charles Morris has 
sought to examine individuals’ ethical and religious orientations 
using a quantitative approach. In his investigations he has de- 
veloped an instrument which he has called “Paths of Life” or 
“Ways to Live.” It consists of 13 paragraphs, each presenting 
a conception of “the good life.” Morris (1956) has epitomized 
these paragraphs as follows: 

Way 1: Preserve the best that man has attained. 


Way 2: Cultivate independence of persons and things. 
Way 3: Show sympathetic concern for others. 

Way 4: Experience festivity and solitude in alternation. 
Way 5: Act and enjoy life through group participation. 
Way 6: Constantly master changing conditions. 

Way 7: Integrate action, enjoyment, and contemplation. 
Way 8: Live with wholesome, carefree enjoyment. 

Way 9: Wait in quiet receptivity. 


Way 10: Control the self stoically. 
Way 11: Meditate on the inner life. 
Way 12: Chance adventuresome deeds. 
Way 13: Obey the cosmic purposes. 
pease ise 
1 Financial support for this research was provided by the University of Cali- 


fornia Committee on Research. Thanks are expressed to Charles Morris for 
his comments on & preliminary draft of the manuscript. 
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The responder’s task in completing the instrument is first to rate 


then to rank them according to his preferences, Using data ob- 
tained in this manner, Morris has explored differences in phi- 
losophies of life according to national groupings, religious af- 
filiation, economic status, temperament, character, body-type, ete. 
His principal findings, together with a copy of the “Ways to Live” 
document, are contained in his book, Varieties of Human Value 
(Morris, 1956). Other Teports of his investigations have appeared 
in psychological Periodicals—e.g., Osgood, Ware, and Morris 
(1961) and Morris and Jones (1955). 


88, Nonetheless lacks the rigor psychologists have come to 
expect in psychometric work. One finds, for example, these state- 


and receptive, then can the wisdom 
should be a serene, confident, quiet 
‘ eat dependable powers which move 
to their fulfillment. The effect of such a mode of expression is diffi- 
cult to assess, (See Winthrop, 1959, for an extensive critique.) 


Problem 


_ The Present Paper is lesg Concerned with the manner of expres- 
sion of ideas, however, than with what may be called the incon- 
gtuity between stimulus complexity and Tesponse simplicity. For 
each of the 13 “supermolar” situations the Tesponder is expected to 
weigh his overall feelings, synthesize his attitudes, and code these 
into a simple number which constitutes his response, Compared to 
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the usual personality, attitude, interest inventory, the scope of 
the input unit here (a paragraph) is unusually large while that of 
the output unit (a number) is standard. In the face of this dis- 
crepaney two focal questions are being raised: (1) Are there con- 
founding elements in the paths which elicit partial responses dis- 
cordant with the response trends to the paragraphs as wholes? 
(2) Can the paragraphs be simplified without destroying the holis- 
tie scope of the document? These may be called questions of homo- 
geneity and simplicity; positive answers to them entail questions 
of reliability which will also be considered here. 


Procedure 


The study was conducted in three phases: a normative or prepa- 
ratory phase, a cross-validation, and a check on reliability. A 
total of 230 Ss, all members of undergraduate psychology classes 
at the University of California, Davis, were used. 

Phase 1. (a) The 13 paths were broken down into 110 sentences, 
some obtained by taking, intact, short sentences from the original 
paragraphs and the remainder by making two or more simple state- 
ments from more complex ones. (b) 24 Ss responded to the original 
document and performed normalized Q sorts of the 110 state- 
ments. (c) On the basis of within-path inter-item 7’s calculated 
from the Q sorts, two other versions of the document were pre- 
pared: (1) a revised form from which empirically discordant ele- 
ments had been eliminated, and (2) a short form containing only 
the most highly intercorrelating elements. 

Phase 2. (a) 40 Ss ranked and rated both the revised and origi- 
nal forms and also made Q sorts of the 110 statements. (b) The 
paragraphs in the original, revised, and short forms were com- 
pared for mean differences in within-path inter-item r’s (part- 
part relationships) and inter-path 1’s (part-whole relationships). 

Phase 3. (a) With appropriate counterbalancing 102 Ss com- 
pleted either two of the three forms or the short form twice, an 
interval of 10 days occurring between tasks. (b) 64 additional Ss 
completed the original, revised, or short form only. (c) Mean test- 
retest and alternate-form reliabilities were estimated, using these 
166 Ss plus the 64 from Phases 1 and 2. 
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Results and Discussion 


a. y 
1. The Question of Discordant Elements. 


On the assumption that even in a small sample, negative cor- 
relation coefficients can be taken as crude indicators of discord- 
ancy, an intercorrelation matrix for each path was derived from 
the information provided by the Q sort in Phase 1. That is, for 
Path 1, which had been separated into 10 statements, 45 r’s were 
computed, the Ss’ Q scores for statement 1 being correlated with 
their Q scores for statement 2, etc., and comparable part-part cor- 
relations were computed between all possible pairs of statements 
in each of the remaining 12 paths. Thus 13 matrices of r’s were 
assembled. 

Tn all but two of the matrices negative coefficients appeared; in 
Some more than half the entries were negative, In Path 4, for exam- 
ple, the statements, The aim of life should not be to control the 
course of the world or society or the lives of others and The 
aim of life should be to be open and receptive to things and per- 
sons, and to delight in them correlated Positively with each other, 
but both showed negative relationships with six of the remaining 
eight statements in the path. No fewer than 42 of the 110 state- 
ments correlated negatively with at least one other statement from 
the same path. 

By eliminating these 42 statements (as many as seven from 
a single path) each of the 13 matrices was reduced to a positive 
manifold. Particularly vulnerable to this type of instrument sharp- 
ening were statements negating or rejecting something, as, for ex- 
ample, Excessive desires should be avoided and moderation sought, 
and A person should not hold on to himself, withdraw from peo- 
ple, keep aloof and self-centered, While of the total of 110 state- 
ments, 35 per cent were negative assertions, in the eliminated 


7 as nt were negations, a disproportion that is 
highly Significant, x? (1) = 20.32, p < .01. 
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for the original. The mean within-path r’s of the statements in the 
original and revised forms were then computed, using the matrices 
in Phase 1 for the former and those in Phase 2 for the latter. These 
results are contained in Columns A and B of Table 1. 

Clearly the homogeneity of those paths from which discordant 
elements were eliminated (Paths 1-11) is improved, the mean r for 
these in the original form being .13, that for the revised, .22. On an 
r to 2 transformation the mean differences in the two sets of 7's is 
highly significant, d: = .09, t (10) = 3.46, p < .01. Despite this 
marked improvement in homogeneity, the within-path relation- 
ships of some of the paragraphs are still very low, e.g., for Path 7, 
the mean inter-item r is only 09. It should be noted that homo- 
geneity, in the sense used above, entails no necessary restric- 
tion on the diversity of stimulus items comprising a given path. It 
is rather an expression of the unity or integration with which 
subjects respond to the path’s varied content. 


TABLE 1 


Mean Within-Path Part-Part* Correlations (r's) for Three Versions 
of Morris's Ways to Live 


Normative Sample Cross-validating Sample 
(N = 24) (N = 40) 
a pacha ase EP Tos 
A B Cc 

Original Revised Short 

Path Form Form Form 
Path 1 09 abt 19 
Path 2 AT .30 36 
Path 3 15 27 87 
Path 4 04 27 67 
Path 5 -20 -18 43 
Path 6 21 26 .38 
Path 7 .04 09 +06 
Path 8 14 16 Sor 
Path 9 .02 .12 —.01 
Path 10 06 127 40 
Path 11 31 34 BL 
Path 12 39 29% AL 
Path 13 38° 25° 61 

Mean of path means: 

Paths 1-11 13 22 spet 
All but Path 8 18 24 -36 

Paths 1-13 17 .23 wave 


a Q sort values of statements comprising paths. 
ò Original and revised forms identical for these paths. 
e In the short form Path 8 a single statement. 


876 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
2. The Question of Simplicity. 


In the” process of constructing the revised form the homo- 
geneity of the paths was increased without deliberate attempt to 
reduce their complexity as stimuli. Ranking them and rating them 
still require a high level of synthesis. The second question raised 
in this study concerns the simplification of the paths, the reduc- 
tion of the number of details to be processed in the task of 
evaluating them. The proposed solution rests on the assumption 
that each of the original paths, even the most heterogencous, con- 
tains some shorter expression that may be taken as a statement of 
its basic significance or essence. If the existence of such key ex- 
pressions can be demonstrated empirically, it should be possible to 
produce an efficient short form of the “Ways to Live” document. 

Two methods suggest themselves for isolating key expressions in 
the paths. The first is simply to select as representative of a given 
path the statement that shows the highest average correlation with 
all the statements in the original paragraph. The second method 
is to select for each path the statement to which Ss respond most 
nearly in the way they respond to the paragraph as a whole, The 
first of these approaches entails the analysis of part-part relation- 
ships, and the second of part-whole. There is no reason to expect 
the two methods to yield identical results, nor indeed to view 
either as intrinsically superior. In the present study both methods 
are used. 

The 13 matrices calculated in Phase 1 were used in the selection 
of key expressions, Within each path the statements were ranked 
in terms of the size of their average intercorrelations. The highest 
ranking statement in each path was automatically included in the 
short form, plus as many additional statements as were needed to 
make the brief path descriptions sufficiently detailed to be distinc- 
tive and roughly equivalent in length. Path 8 is composed of one 
long statement, Path 2 of four short ones; the remaining paths con- 
tain either two or three. Morris's original wording was adhered to 
as closely as possible. The short form, brief enough to be repro- 
duced on a single 844 X 11 sheet, is shown in Fi 

y) own in Figure 1. 

As in the revised form, the items in the short form were cross- 
heya, on the 40 Ss in Phase 2. To check part-part relation- 

ps in this form, the appropriate inter-item coefficients were se- 


ji a RHE | 
HEE aaam atin la lif 
HE i HE l ttl i HiL pil i 


El 
S Sas 3 = 3i iy 
RE i ia HR 
ot 5 e i s FFER asfi 
uy pa nainn piit HEE 
HIERE Hiki eededbeieed ls ities 
Freee aca a Bree le nip fate 
ie a HEE Ka i i 
SENTIRE GDE nS ah 
Ee EJE OE KELD SEERE A pps 
HER Lr nue aa iki i iang 
M TT a 
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for it is the active deed that is satisfying, the deed adequate to the 
present, the daring and adventurous deed. 

Path 18. One should let himself be used by other persons in their 
growth, and by the great objective purposes in the universe. One should 
be a serene, confident, quiet vessel, guided by the great dependable 
powers which silently and irresistibly achieve their goal. 


lected from the within-path matrices, and the mean r determined 
for each path (Table 1, Column C). The mean within-path correla- — 
tion of the short form is .36, a figure that compares favorably with 
a corresponding mean of .18 for the original form and .24 for the 
revised form (Table 1). Just as the homogeneity of the revised” 
form was significantly greater than that of the original, the homo- = 
geneity of the short form exceeds that of the revised, d; = .14, ue 
(11) = 2.50, p < .05. In terms of part-part relationships, then, the 
paragraphs of the short form appear to represent a more consist- 
ent core of meaning than either the original or the revised form. 
Despite this overall improvement, however, it is to be noted that 
two paths (7 and 9) are still as heterogeneous as they were orig- 
inally. 

To examine part-whole relationships, the Q sort values for each 
of the 110 items were correlated with the 40 Ss’ ratings of the 
original paths. The range of these 110 r’s was large, from —.33 to 
+.67. As in the analysis of part-part relationships, three separate 
mean 1’s were calculated for each path, one based on all the items 
in the original paragraph, a second on the items in the revised | 
path, and a third on the items in the short path. These means are | 
presented in Columns A, B, and C, respectively of Table 2. Each 
progressive change in the composition of the paths is accompanied 
by a highly significant improvement in mean part-whole relation- 
ships. For the 11 paths modified from the original to the revised 
form, the mean r rises from .23 to 30, d, = .08, t (10) = 6.67, 
p < 01. All 13 paths were altered from the revised to the short 
form and the comparable figures are .31 for the revised and .39 for 
the short, dı = 09, ¢ (12) = 3.91, p < .01. Again it should be — 
noted that even with the progressive improvement in part-whole 7’s, 
Paths 7 and 9 remain, as in the part-part analysis among the least 
homogenous paths. 
ine ane meen in both part-part and part-whole re- 

at the paragraphs of the short form taken 
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individually express the core conceptions of Morris’s document 
more adequately than the original paragraphs themselves. 


TABLE 2 


Mean Within-Path Part-Whole* Correlations (r's) for Three Versions of Morris's 
Ways to Live as Shown in Cross-Validating Sample (N = 40) 


3 A B Cc 

Original Revised Short 
Path Form Form Form 
Path 1 -26 .32 46 
Path 2 +29 +39 50 
Path 3 24 84 42 
Path 4 17 31 .38 
Path 5 .22 .34 45 
Path 6 17 +23 14 
Path 7 12 -19 27 
Path 8 34 36 48 
Path 9 -10 17 -18 
Path 10 26 32 85 
Path 11 -26 30 +85 
Path 12 445 445 48 
Path 13 33° +835 52 

Mean of path means: 

Paths 1-11 +23 -30 36 
Paths 1-13 +25 31 89 


a aaastal 
«Part = (Qsort values of statements comprising paths) vs. Whole = (Ratings of original 


paragraphs). 
b Original and revised forms identical for these paths. 


3. The Question of Reliability. 


While the complexes of items comprising the revised and short 
forms have been shown to be more homogencous and simpler than 
those of the original, the reliability of each form as a separate 
document has not yet been explored. In the final phase of the pres- 
ent study estimates of test-retest and alternate-form relation- 
ships were obtained both across paths and across individuals, spe- 
cial attention being paid to the short form. 

Some of the reliability coefficients reported by Morris (1956) 
appear to be across-path estimates, a form of estimate suitable for 
the group comparisons he was concerned with, but one that is 
hardly appropriate for the study of individual differences. Using 
Ss’ ratings, Morris estimates the repeat reliability of his document 
to be about .85 for college students; he reports average coefficients 
of 87 and .78 from two specific studies. The across-path reliabil- 
ity coefficients reported in the present study are Spearman rho’s 
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based on mean ranks of the paths as shown in six independent 
samples ranging in size from 32 to 46 Ss. For each form two sam- 
ples were used, thus making available a single repeat reliability 
estimate for each form and four alternate-form estimates for each 
pair of them. Repeat estimates are uniformly high, .93, .95, and 
-96 for the original, revised, and short forms, respectively. The 
mean alternate form estimates are lower: .83 between original and 
revised forms, .82 between revised and short, and .71 between 
original and short. These lower coefficients reflect changes in 
the overall acceptability primarily of two paths as a consequence 
of the refining process. The contemplative life (Path 11) becomes 
more attractive with the elimination of statements like The ez- 
ternal world is no fit habitat for man; it is too big, too cold, too 
pressing. Conversely, the enjoyment of simple pleasures (Path 8) 
becomes relatively less attractive when the discordant elements 
have been removed from the more active, social, and achievement- 
oriented paths. 

The analysis of across-individual relationships also reveals a 
relatively high level of consistency. For each person who took 
two forms of the document, or the short form twice, a Spearman 
tho was calculated from his two rankings. On the basis of 2’s, 
mean rho’s were as follows: for the short-short sample the mean 
tho was .80 (N = 82); for the short-revised sample, .72 (N = 
35); for the revised-original sample, .63 (N = 40); and for the 
short-original sample, 57 (N = 35). The mean rho of .80 is some- 
what higher than the reliability estimate of .67 (mean Pearsonian 
r based on individual repeat ratings) reported by Morris and 
Jones (1955). 

On the basis of the mean test-retest coefficient of .80 it is evident 
that the path descriptions in the short form are sufficiently distinc- 
tive to be ranked reliably. This, of course, does not imply that 
every S's performance is consistent; in each of these samples a few 
individual Coefficients were at the zero level. Since inconsistent 
performers cannot be identified on the basis of a single administra- 
tion, it 1s recommended that repeat rankings be obtained when- 
ever individual performances are a matter of concern. 

The mean alternate-form coefficient of -72 between the short and 
revised forms was not significantly different from the repeat re- 
liability of the short form itself, d; = 20, t (65) = 1.67, p > .05. 
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Within the limits of these reliabilities, these forms may be taken 
as equivalent. With the original form, however, the short form 
shows a mean coefficient (.57) that is significantly lower than its 
repeat reliability, d; = 46, t (65) = 3.83, p < .01. Thus, while 
there is a substantial relationship between the original and short 
forms, there are also important differences between them, As 
with the across-path results, these differences reflect systematic 
changes in the acceptability of a few paths following the elimina- 
tion of certain statements. The differences noted earlier for Path 
11, the contemplative path, also appear on the individual level, 
for example, and the place of enjoyment is again a focus for 
down-grading. This time, however, the most obvious shifts are in 
the reduced acceptability of festivity and sensuous enjoyment 
(Path 4) rather than of the simple, easily obtainable pleasures 
(Path 8). 

A comparison of the content of the short form with that of the 
original indicates that Morris’s a priori judgment as to what was 
central in the various paths was remarkably accurate. His intro- 
ductory statements or topic sentences in ten of the 13 original de- 
scriptions emerged as key expressions from the empirical analysis 
as well. Between Morris’s brief characterizations of the paths 
and the implications of the data of this study, however, some 
notable differences do occur. Path 4, for example, Morris believed 
to emphasize festivity and solitude in alternation. Analytically it 
emphasizes festivity and sensuous enjoyment, solitude being ir- 
relevant. In the present analysis Path 7 appears to be purely eclec- 
tic, emphasizing diversity and reflecting not at all the integration 
Morris posited as central. Also, in Path 12, the use of the body’s 
energy plays a far more important part empirically than Morris's 
characterization of it as chancing adventuresome deeds would im- 
ply. These and other less notable differences suggest the following 
as a more accurate epitome of the paths than that given by 
Morris (1956) in his book and cited in the opening paragraph 
of this paper (alterations in italics) : 

Way 1: Appreciate and preserve the best man has attained. 

Way 2: Cultivate independence and self-knowledge. 

Way 3: Show sympathetic concern for others. 

Way 4: Experience festivity and sensuous enjoyment. 

Way 5: Actand enjoy life through group participation. 
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Way 6: Master threatening forces by constant practical work. 

Way 7: Admit diversity and accept something from all ways of 
life. 

Way 8: Enjoy the simple, easily obtainable pleasures. 

Way 9: Waitin quiet receptivity for joy and peace. 

Way 10: Control the self and hold firm to high ideals. 

Way 11: Meditate on the inner life. 

Way 12: Use the body’s energy in daring and adventurous deeds. 

Way 13: Let oneself be used by the great cosmic purposes. 


Summary 


The 13 paragraphs of Morris’s document were divided into 110 
statements, Q sorts of which were used to assess the coherence of 
his complex stimulus-situations. Eleven paths showed discordancy 
(negative within-path inter-item r’s), the bulk of the negative 
relationships stemming from “thou shalt not” statements rather 
than from assertations of positive value. A revised form (discord- 
ant statements omitted) showed improved homogeneity in cross- 
validation, and a short form (composed of empirically-derived 
key expressions) showed still higher homogeneity. Across-path re- 
liability is high in all forms. Alternate-form comparisons indicate 
equivalence of short and revised forms, with some systematic 
changes from the original. Test-retest reliability of the short 
form is .80. Ss were 230 college students. 
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TEST-RETEST RELIABILITY OF THE EPPS! 


DANIEL V. CAPUTO 
Queens College (CUNY) 
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Washington University, St. Louis 


In his manual for the use of the Edwards Personal Preference 
Schedule, (EPPS), Edwards (1959) reported test-retest relibility 
coefficients for each of the 15 need scores plus a consistency score 
for a group of 89 University of Washington students who were re- 
tested after a one week interval. Subsequent studies have reported 
test-retest reliabilities of need scores after intervals of three weeks 
(Mann, 1958) and one week (Horst and Wright, 1959). Coefficients 
from these studies are reported in Table 1. Borislow (1958) has also 
reported test-retest reliabilities ranging from .65 to .91 for the pro- 
file of need scores for Ss after a two week interval. 

All these studies have demonstrated satisfactory short-term re- 
liability for the test and consequently call into question the argu- 
ment of Levonian, Comrey, Levy, and Procter (1959) who stated 
that the basic form of the EPPS item is one that encourages low re- 
liability of response since the S must choose which of two state- 
ments seems more descriptive of himself while at the same time this 
choice is made more difficult by equating the statements for social 
desirability. This procedure maximizes the number of difficult and 
hence potentially unreliable choices for the S. 

As part of a broader study of the EPPS used with nursing stu- 
dents, it is possible to report reliability coefficients for a much 
longer inter-test period than previously investigated. The EPPS 
was administered to a group of 79 female freshman nursing stu- 
This paper is a partial report of a research project “Role Differentials and 
Nursing Ideology,” John A. Stern and Albert F. Wessen, Co-Principal Inves- 
tigators, PHS Grant NU-00050. A portion of the computation was done with 


support provided by the Washington University Computer Center under Na- 
tional Science Foundation Grant G-22296. 
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dents in November, 1962, and readministered to the 52 students 
who were still in school in March, 1964. Test-retest product- 
moment correlations were computed for each of the EPPS needs 
across the 52 Ss who had taken the test at both administrations. 
These coefficients and the corresponding means and standard de- 
viations are reported in Table 1. Profile intercorrelations were also 
determined for each student by correlating her profile of 15 need 
scores at time 1 with her profile at time 2. The distribution of these 
correlation coefficients is shown in Figure 1. 

The test-retest correlations between needs in the present study 
are all significantly greater than zero. However, all are lower than 
those obtained in the studies which used a one-week interval (see 
Table 1). Use of z-tests to compare the correlations of the present 
study with those of Edwards’ reliability study showed that 11 of 15 
of the present correlations are significantly lower. With the excep- 
tion of Affiliation and Nurturance, whose test-retest correlations 
are higher in the present study, all are also lower than those ob- 
tained by Mann in his three-week interval study. Thus, the corre- 
lations between Ss’ scores on the individual EPPS needs before and 
after a 15-month interval are significantly positive though not as 
high as are comparable correlations obtained using shorter test- 
retest intervals. 
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Examination of the correlations obtained between the individual 
profiles of Ss before and after the 15-month interval (see Figure 1) 
reveals an approximately normal distribution. The mean of the 52 
profile correlations is .66, and is significantly greater than zero 
(p < .05). Fifty-one of the profile correlations are significantly 
greater than zero at the .05 level, while 48 of these also attain sig- 
nificance at the .01 level. The one S whose profile correlation 
(r = .03) is not significant obtained a Consistency score of 9/15 on 
the second test administration which suggests that her responses at 
that time may have been made randomly (Edwards, 1959). Boris- 
low’s individual profile correlations, obtained using a two-week 
test-retest interval, are also shown in Figure 1. Five of the six are 
greater than the mean of the present profile correlations. 

The possibility of significant differences existing between the pre 
and post 15-month interval need scores of the subjects of the pres- 
ent study was investigated by applying t-tests for correlated means 
to the need scores at time 1 and time 2. As shown in Table 1, needs 
Deference, Affiliation, and Nurturance were significantly lower at 
the second testing time, while needs Dominance and Heterosexual- 
ity were significantly higher. 

The question may be raised as to whether a retest after a long 
time interval such as the present one represents an investigation of 
test reliability or subject change. Obviously, both may be involved. 
However, from the perspective of test reliability it can be con- 
cluded that over the long term, the EPPS shows acceptable reliabil- 
ity both in regard to measurement of a single scale over all Ss and 
for a single S over all scales, 
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CERTAIN CONSEQUENCES OF APPLYING THE 
K FACTOR TO MMPI SCORES 


GEORGE D. YONGE 
University of California, Davis 


Tux present study is focused on some of the psychometric con- 
sequences of using the K scale to correct certain Minnesota Multi- 
phasic Personality Inventory (MMPI) clinical scales for distor- 
tions induced by test taking attitudes. 

There is a growing bibliography documenting the ineffectiveness 
of using the K correction as a means for improving the clinical 
validity of certain MMPI scales (see Dicken, 1963). At the same 
time, it is instructive to note that the psychometric consequences of 
using the K scale as a correction factor have seldom been evalu- 
ated apart from focusing on external criteria. This is not surprising 
in that the philosophy underlying empirically derived tests, such as 
the MMPI, places a premium on the ability of the test to correlate 
with a criterion. Any question of a logical or psychological nature 
is clearly subordinate to this concern for correlations with criteria. 
A clear statement of this philosophy is provided by Fricke (1963) 
who essentially states that the characteristics of a test—reliability, 
content, ete—are not important as long as the test predicts the 
criterion. There would be no argument with this philosophy if the 
test has a high correlation with a criterion (which is seldom the 
case) and if the psychologist is content to make no psychological 
interpretations of the test scores. 

It is contended that most psychologists use tests such as the MMPI 
to learn something about individuals rather than to blindly predict 
criterion scores or group membership. To the extent that this con- 
tention is true, questions must be asked regarding the logical and 
psychological aspects of the measurement operations performed. 
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When the use of the K score as a correction factor is evaluated 
from a, logical, psychological, and psychometric perspective, there is 
little justification for its use. 

One study which raises some psychometric questions in connec- 
tion with the use of K is that by Tyler and Michaelis (1953). These 
authors formulated six questions to be considered in evaluating the 
use of the K scale as a correction for response distortion. Whereas 
Tyler and Michaelis addressed themselves to four of these ques- 
tions, the present study is addressed to two of the four treated by 
Tyler and Michaelis and one not asked by them. 

Specifically, the questions! asked in the present study are: 

1. What are the correlations between the K scores and scores on 

the clinical scales to which K is applicable? 

2. What are the correlations between the K-corrected and uncor- 

rected scores? 

8. Does the K correction affect the correlations of the MMPI 

scales with other measures of social-emotional adjustment? 


Method 


One hundred and thirty-six? University of California students 
applying for a study abroad program were given the Omnibus Per- 
sonality Inventory (Center for the Study of Higher Education, 
1962) and the Minnesota Multiphasic Personality Inventory 
(Hathaway and McKinley, 1951). The answers to the above three 
questions were approached by means of Pearson product-moment 
correlations. Although not reported, the essential results of the 
present study were also obtained with a sample of 58 University of 
California Peace Corps applicants. 


Results 


1. What are the correlations between K scores and the scores on 
the clinical scales to which K is applicable? 


The answer to this question, in part, underlies the answers to the 
subsequent questions. Table 1 presents the correlations between the 
K scores and the scores derived from the relevant clinical scales. 

1 Questions 1 and i i i 
pre es io ap aed Spa treated by Tyler and Michaelis; ques- 


2 The data on which the present study i i 
Barbara Kirk, University of California Ccumeeing Gens Ptrined from Mrs 
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For comparative purposes, the results obtained by Tyler and 
Michaelis are also reported. 

The correlations between K and the relevant clinical scores are 
remarkably similar to those obtained by Tyler and Michaelis. It is 
also interesting to note that there is a rough correspondence be- 
tween the magnitude of these correlations and the proportion of K 
recommended to correct a given scale.* It is curious, however, that 
no correction is recommended for the Hysteria (Hy) scale, which 
correlates .39 with K, or for the Social Introversion (Si) scale, 
which correlates —.55 with K. If indeed K is a measure of response 
bias, Hy and Si are about as permeated with this unwanted vari- 
ance as are the scales for which a K correlation is recommended. 

2. What are the correlations between the K-corrected and wncor- 
rected scores? 

Table 2 presents the data relevant to this question. Again, for 
comparative purposes, the results obtained by Tyler and Michaelis 
are also reported. 

The high correlations between the corrected and uncorrected 


TABLE 1 
Correlations between K and the MMPI Scales to Which K is Applicable 
ee ee A 


MMPI Scales 
inp eee 
N Hs Pd Pt Se Ma 
Present Study 136. —.414*  —.28* —.75* —.638% = .89* 
Tyler and Michaelis 56. —.46" «—.24 69% 50" = .87* 


* Significant .01 level. 


TABLE 2 
Correlations between K-Corrected and Uncorrected MMPI Scores 
Correlated Present Study Tyler and Michaelis 
Scores (N = 136) (N = 56) 
Hs - Hs + 5K .55* .66* 
Pd + Pd + 4K -90* .91* 
Pi -Pit +K .64* .67* 
Sc + Sc +K .58* .56* 
Ma. Ma + .2K .98* .97* 


a 
* Significant .01 level. 


EA Oe NESE 
3 The recommended proportions of K are: Hs + 5K, Pd + 4K, Pt + 1.0K, 
Sc + 7.0K, and Ma + 2K. 
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scores for Psychopathic Deviate (Pd) and Hypomania (Ma) indi- 
cate that the K correction has a minor effect upon these two scales. 
This finding is not surprising and is no doubt a reflection of the low 
correlations these scales have with K and the small proportion of 
K used to correct these scales. Tyler and Michaelis, noting that the 
reported reliabilities of Pd and Ma are considerably lower than the 
corresponding corrected-uncorrected correlations, concluded that 
the uncorrected scores are apparently better measures of the cor- 
rected scores than of themselves. Unfortunately, Tyler and Mi- 
chaelis have overlooked the spurious nature of these correlations 
owing to the part-whole correlation involved. The spurious nature of 
the correlations presented in Table 2 is a strong recommendation 
against comparing them with the reliabilities of the scales. 

The point to be made from the correlations in Table 2 is that the 
K correction markedly alters whatever is being measured by Hypo- 
chondriasis (Ha), Psychasthenia (Pt), and Schizophrenia (Sc). 
Furthermore, these correlations underplay the change of measure- 
ment which has taken place simply because they are spurious corre- 
lations. 

A consideration of the data presented in Table 2 gives rise to a 
question asked but not treated by Tyler and Michaelis; namely, 
how do the reliabilities of the corrected scores compare with those 
of the uncorrected scores? Since this question can be treated only in 
a tangential manner in the present study, it was not raised as a 
major question. 

Kuder-Richardson Formula 21 reliability coefficients were com- 
puted for the corrected and uncorrected Pt and Sc scores. These 
two scales were used for this analysis because the entire K score is 
added to each of these scores as the recommended correction. Thus, 
by correction, the length of these scales is increased by approxi- 
mately 30 items.* However, as indicated in Table 3, the Kuder- 
Richardson reliabilities change from high positive to low negative 
values. Thus, from the point of view of internal consistency relia- 
bility, the measurement attained with the uncorrected scales is 
re ere out when the K correction is applied. It should 
NO a e data in Table 3 offer no implications regarding 

st reliabilities. In fact, Rosen (1953) has reported that 


4 The item overlap between K and Pt is two, between K and Sc one. 
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with a sample of 40 psychiatric patients retested after four days 
(on the average) the test-retest reliabilities of Pt and Sc increased 
slightly when the K correction was used. 

3. Does the K correction affect the correlation with other meas- 
ures of social-emotional adjustment? 

It will be remembered that the subjects in the present study were 
also administered the Omnibus Personality Inventory (OPI). There 
are three measures in the OPI which presumably focus on social- 
emotional adjustment. These are Impulse Expression (IE), Schiz- 
oid Functioning (SF), and Lack of Anxiety (LA). The correlations 
between these OPI measures and the corrected and uncorrected 
MMPI scores are presented in Table 4. 

The most striking feature of the data presented in Table 4 is the 
consistent reduction in the OPI-MMPI correlations when the K 


TABLE 3 


Means, Standard Deviations, and Kuder-Richardson Formula 21 Reliability 
Coefficients of K-Corrected and Uncorrected Psychasthenia 


and Schizophrenia Scores 
Uncorrected Corrected 
eae ANR ao aaa Bas 
KR 21 Mean §.D. KR 21 Mean &.D. 
Pt 78 8.5 5.5 — .28 26.5 3.7 
Se 73 8.1 5.1 —.18 26.2 4.1 
TABLE 4 


Correlations between Selected OPI Social-Emotional Adjustment Scales 
and K-Corrected and Uncorrected MMPI Scales 


OPI Scales 

MMPI Scales IE SF LA 
Hs 14 .40* —.37* 
AN E A MRT 
Be Paes ink at) E 
ee eae ct 

.47* :41* —.30* 
Ma + 2K A Ue 


* Significant .01 level. 
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correction is applied. Particularly, the reduction in the Pt-SF, Pt- 
LA, Sc-SF, and Sc-LA correlations may be interpreted as indicat 
ing that the K factor reduces the validity of these MMPI scales. 
This consequence of using the K correction is, of course, consistent 
with the other data presented in the present paper. 


Discussion and Conclusions 


As a measure of test-taking attitude, the K scale represents a 
concerted effort to assess a person’s defensiveness or tendency to 
place himself in a good light. In view of the socially undesirable 
content of the clinical scales, the defensiveness reflected in the K 
scale will presumably “hold down” a person’s scores on the clinical 
scales, Therefore, by adding K (Defensiveness) to Pt, for example, 
the “defensiveness” variance in Pt will be cancelled out (since K 
and Pt correlate negatively, adding K amounts to a subtraction or 
reduction of variance), This assumes that Psychasthenia and de- 
fensiveness are additive components present in the uncorrected 
score. A parallel argument in the area of achievement testing would 
have the psychometrist add to the raw score a measure of a tend- 
ency to make careless errors in order to obtain a more accurate 
estimate of the subject’s level of achievement. Since it is recognized 
that errors, careless or otherwise, are aspects of the person’s 
achievement status, we do not find this operation applicable to the 
achievement domain. And, in fact, if a measure of poor achieve- 
ment (errors) were added to a measure of good achievement (cor- 
rect responses), it would seem that the content of the measure 
would be washed out rather than purified, The data presented in the 
present study are interpreted as indicating that the K correction 
nullifies rather than purifies the measurement obtained with the 
uncorrected scales. 

_ Within the perspective of the above discussion, a further con- 
sideration of the Pt and Sc scales, to which the entire K score is 
added as a correction, will point to the unusual logic and conse- 
quences connected with the use of the K factor. 
spe a a an Laas where the correlations are —.75 and 
ing that this procedine PTER RETA Te : aa co. 
Pate N, purify the Pt and Se dimensions. Op- 

Y, this is similar to adding correct and incorrect answers 


GEORGE D. YONGE 893 


on an achievement test; this would tend to reduce the scale vari- 
ance and virtually cancel out the content of the test. By adding K 
to Pt and to Sc (ie., increasing their lengths by 30 items) their re- 
spective standard deviations are decreased from 5.5 to 3.7 and from 
' 5.1 to 4.1! This is analogous to what would be expected if correct 
and incorrect answers to an achievement test were added together. 

When there is evidence that a person’s scores are influenced by 
test taking attitudes, it is difficult to see how intra-test manipula- 
tions of these biased scores will somehow rid them of their bias. 
Rather, it would seem more reasonable to assess the person’s per- 
sonality, attitudes, etc., by some other measurement operation. 

The array of data considered in the present paper converge on 
one general conclusion. The application of the K factor, from a 
psychometric viewpoint, considerably alters the structure of the 
MMPI scales. Specifically, the K correction reduces the internal 
consistency reliability and the validity of these measures. The 
above findings, in conjunction with the growing body of negative 
evidence concerning the utility of the K correction in increasing the 
clinical validity of the MMPI, clearly argue against the continued 
use of K as a correction factor. 
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Tum Vanity Srupms Section is published twice a year, once in 
the Summer issue and again in the Winter issue, for which the clos- 
ing dates for receiving manuscripts are February first and August 
first, respectively. Although articles between two and eight printed 
pages are usually preferred, an occasional exception is made to pub- 
lish articles of somewhat greater length. 

Considerable flexibility exists concerning format as can be seen 
from a study of recently published articles. However, the model 
presented in the Spring, 1953, issue of EDUCATIONAL AND PsycHo- 
LOGICAL MEASUREMENT still represents a close approximation to 
what is customarily published. The prospective contributor is en- 
couraged to read the original announcement. 

In order that the usual number of articles of other types may not 
be reduced, it is necessary to enlarge the journal and to charge the 
authors for most of the publishing costs. Due to the financial pres- 
sure of rising printing costs, the new rate for publication of articles 
in this section, beginning with the first issue for the year of 1967, 
will be twenty-five dollars per page. The extra cost of the composi- 
tion of tables and formulas will be added to the basic rate. One 
hundred gratis reprints is extended to the author of articles ap- 


pearing in this journal. 
Two copies of manuscripts should be sent to: 
Dr. William B. Michael 
Professor of Education and Psychology 
University of California, Santa Barbara 
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NONINTELLECTIVE PREDICTORS OF ACHIEVEMENT 
IN COLLEGE* 


ROBERT C. NICHOLS 
National Merit Scholarship Corporation 


Spxnotion of students in college admissions and scholarship pro- 
grams is usually based in part on a prediction of the student’s per- 
formance in college. The accepted predictors are aptitude test scores 
and high school grades, often combined into a single index by a re- 
gression equation. The conventional criterion for validation of these 
predictions is the college grade point average, usually for the fresh- 
man year. There is a well developed technology for making these 
predictions, and many colleges and scholarship agencies have 
developed formulas for optimum weighting of high school grades 
and test scores in their local situation. 

The growing uniformity of selection practices might lead one to 
assume that the prediction of college performance has reached a 
state of stable maturity satisfactory to most practitioners. Instead, 
there is increasing concern and dissatisfaction with the current state 
of the art. This concern may be, in part, a reaction to the recently 
vocal lay critics of the testing movement, but there is also a recog- 
nition of the inadequacy of current selection methods to meet the 
demands being made of them. There are two major sources of dis- 
satisfaction: 

1. The popular colleges and most scholarship and honors pro- 
grams have many more applicants than they can accept. After ap- 
plicants with low grades or low test scores are eliminated, further 
discriminations must be made between the remaining highly quali- 


NEEE 
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fied candidates. The selection committee can make these decisions 
on the basis of the remaining small differences in test scores and 
high school rank; they can look at other data, such as an inter- 
yiewer’s impression, a recommendation or an autobiography; or 
they can flip a coin. These three methods of discriminating be- 
tween students with good grades and high test scores are about 
equally valid when judged by the criterion of later student perform- 
ance. Educational researchers have been busy searching for indica- 
tors of success other than grades and test scores (usually called 
non-intellective predictors) and some have been found. However, 
none has yet achieved enough demonstrated success to be widely 
adopted. 

2. There is growing dissatisfaction with the use of college grades 
as the criterion for evaluation of predictors. Some educators are 
beginning to feel that the student who makes the best grades is not 
necessarily the most valuable student. For example, Stalnaker 
(1965) in discussing the National Merit Scholarship selection pro- 
gram, said, “. . . we want to find students who will succeed in col- 
lege, but—much more important—will also use their college educa- 
tion in some socially desirable, productive way after graduation. 
How relevant are grades to this goal? ... Do you inquire of your 
accountant, your physician or your lawyer the grades he received 
in college? Predicting grades has little social significance” (p. 134). 
Holland and Richards (1965) have pointed out that a student's 
extracurricular achievement may be more similar to achievement 
after graduation than is the academic achievement represented 
by grades. They demonstrated that academic and extracurricular 
achievement in college are not highly correlated and urged greater 
use of extracurricular achievement as an alternative criterion to 
grades in the development of selection devices. Hoyt (1965) con- 
cluded from a review of relevant studies that there is little relation- 
ship between college grades and post-college achievement. 

The present study grew out of these two sources of dissatisfaction 
with current methods of predicting success in college. The study has 
two main goals: First, to find nonintellective predictors that will 
make effective discriminations between students who have already 
been highly selected on test Score; and second, to find predictors of 


extracurricular achievement which will, hopeful] i 
of the predictors of grades. lanai 
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Questionnaire materials were mailed to a sample of 1843 Na- 
tional Merit Finalists and National Merit Scholars in the spring of 
1962, shortly before their expected graduation from high school, 
and also to a sample of 1383 students chosen at random from all 
students who took the National Merit Scholarship Qualifying Test 
(NMSQT) in 1961—the same time that this test was taken by the 
sample of Merit Finalists and Scholars. Usable returns were ob- 
tained from 86 per cent of the Merit Finalists and Scholars (the 
Merit sample) and from 64 per cent of the random sample of 
NMSOQT participants (the normative sample). 

A second questionaire was sent to the respondents to the first sur- 
vey in the summer of the following year, when the students in the 
sample who attended college should have finished their freshman 
year. This questionnaire inquired about their college grades, their 
extracurricular achievement and some additional items for another 
study. Returns were obtained from 92 per cent of the Merit sample 
and from 84 per cent of the normative sample. Those who did not 
attend college for the entire year and those with missing data were 
discarded leaving a Merit sample of 1330 and a normative sample 
of 419 for the study. 


Predictors 


The first questionnaire obtained student responses to four groups 
of personality, attitude, interest and behavior items: (a) the Cali- 
fornia Psychological Inventory, a well known personality inventory 
consisting of 480 True-False items (Gough, 1957), (b) the Voca- 
tional Preference Inventory (VPI), an interest inventory consist- 
ing of 160 occupational titles to which the student responds Like- 
Dislike (Holland, 1958, 1965), (c) the Adjective Check List (ACL), 
adopted with some additions and deletions from a checklist de- 
veloped by Gough (1960) and in which the student checks from a 
list of 159 adjectives those he considers descriptive of himself, and 
(d) the Objective Behavior Inventory (OBI), developed for this 
study and consisting of 326 things that a student might do (hobbies, 
sports, leisure time activities, interaction with others, ete.). The 
student indicates whether he has engaged in each activity fre- 
quently, occasionally, or not at all during the past year. In con- 
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trast to the other three inventories, which inquire about attitudes, 
opinions, and feelings that are known only to the student himself, 
the OBI items refer to overt behavioral acts which could be 
seen by an external observer. 

Criteria 

Two criteria of college performance were used—academic achieve- 
ment and extracurricular achievement: (a) Academic achievement 
was assessed by the student’s self report of his freshman grades on 
a letter grade scale (A, A—, B+, B, B—, ete.). Nichols and Holland 
(1963) compared such self reported grades with grades calculated 
from a transcript and found the self reports to be quite accurate 
(r = .96). Davidsen (1963) found similar validity (r = .92) for 
self-reported high school grades obtained in a selection context. 

The students attended a variety of colleges, and it seems likely 
that a given grade may mean quite different things in different col- 
leges or even in different departments within a single university. To 
the extent that this is the case the grade criterion will be less pre- 
dictable from student characteristics assessed before college entry. 
Thus, we would not expect to find validity coefficients as high as 
might be obtained when predicting them at a specific college. How- 
ever, a sample of students attending many colleges offers two 
specific advantages: only predictors that have validity in a variety 
of college environments will be found; and the validity coefficients 
will give an indication of the practical utility of the predictors in a 
setting where predictions must be made before the student’s col- 
lege is known. 

The main danger in combining students from different colleges is 
that the students grades may be determined in part by the quality 
of competition from other students at the college. Thus, the analysis 
may turn up correlates of attending selective colleges rather than of 
grade getting behavior per se. To check this possibility an ad hoc 
sample of 2,000 students attending 246 colleges was drawn from & 
much larger number available from another study (Astin, 1965; 
Werts, 1966). The partial correlation of the student’s freshman 
grade average with the selectivity of the college he was attending 
was —.08 with the student’s high school grades partialled out, —-12 
with the student’s NMSQT score partialled out, and —.18 ith both 
high school grades and NMSQT score held constant statistically- 
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The index of selectivity was the number of National Merit Com- 
mended students choosing the college divided by the number of 
freshmen admitted (Astin, 1965). The possibility of adjusting col- 
lege grades for college selectivity was considered, but was not done 
in this study because the variance in selectivity was not great 
among the colleges attended by the Merit sample, the relationship 
of selectivity to grades was not large, and there is no really ade- 
quate method of equating grades across colleges. (b) Extracurricular 
achievement was assessed from the student’s report of his non- 
academic activities during the freshman year. The questionnaire 
asked the student to describe all achievements during the past 
year in the following areas: Leadership, Science, Art, Music, 
Writing, Speaking, Dramatics, and Athletics. Examples were given 
of possible achievements in each area and space was provided 
for the student to write in his achievements. This free response pro- 
cedure is felt to be an improvement over the checklists used in 
previous studies (Nichols and Holland, 1963; Holland and Nichols, 
1964)—the possibility of error from random and erroneous check- 
ing of checklists of very rare events was reduced, and many 
achievements were obtained that had not been included on the 
checklists. 

The responses in each area of achievement were coded on a three 
point scale: (1) no achievement, (2) achievement not involving 
outside recognition, (3) outstanding achievement receiving outside 
recognition of quality. These achievements were combined into a 
single three point extracurricular criterion scale by counting those 
students with no achievement in any area as nonachievers (scored 
1), those with one or more outstanding achievements as achievers 
(scored 3), and the rest in between (scored 2). The few students with 
recognized leadership achievement, but no other achievement, were 
scored 2 instead of 3 to keep the criterion scale from being too 
heavily weighted with leadership. The extracurricular achievements 
were combined into a single scale because there were not enough 
outstanding achievers in the individual areas and the lower level 
achievements were not of sufficient social significance to warrant 
separate study. However, combining the achievements is justifiable 
on both empirical and logical grounds: achievements in the various 
areas tend to have low positive correlations with each other. Since 
time limitations make it difficult for a student to achieve in more 
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than one area, the low positive correlations seem to indicate a sub- 
stantial general tendency to achieve. Roberts (1965) developed 
scales to predict specific extracurricular achievements and found 
similar content among the scales. His scales developed to predict 
achievement in one area also predicted achievement in other areas, 
and his achievement scales had much higher intercorrelations 
among themselves than did the actual achievements. Potential 
users of predictors of extracurricular achievement are more con- 
cerned with a general tendency to achieve than with the specific 
area of achievement, because, like grades, extracurricular achieve- 
ments in college are of value mainly as a demonstration of a tend- 
ency to achieve which is expected to persist to some degree after 
college. 


Analyses 


Scales were derived by item analysis against grades and extra- 
curricular achievement using a portion of the Merit sample. After 
excluding students who did not attend college for a full year and 
those with incomplete data, 1013 Merit students (those with ID 
numbers ending with a digit less than seven) were assigned to the 
derivation group and the remaining 419 to the cross-validation 
group. No special separation of the sexes was made. 

For item analysis against the grade criterion high and low groups 
of approximately the upper and lower 27 per cent on first year col- 
lege grades were used: the 262 students who reported a first year 
grade average of A or A— and also reported receiving some recog- 
nition for academic achievement (dean’s list, honor society, etc.) 
composed the high group. The low group consisted of the 246 stu- 
dents with a first year grade average of B— or lower and who re- 
ceived no recognition for academic achievement. For item analysis 
against the extracurricular achievement criterion, the high group 
consisted of the 229 students who received outside recognition for 
one or more extracurricular achievements and the low group con- 
sisted of the 294 nonachievers, 

Phi coefficients were computed for each of the 1125 items against 
each criterion. The obi items, which used a three alternatives re- 
sponse format, were dichotomized by combining the middle cate- 
gory with the smallest extreme category. Separate academic and 
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extracurricular achievement scales for each of the four item formats 
were formed by combining all items significant at the .05 level. The 
validity of these scales and their usefulness in combination with 
test scores and high school rank were checked using the cross- 
validation group of Merit students and the normative sample. 


Results 


The characteristics of the scales derived from the four item pools* 
are shown in Table 1. Of the major item types the OBI and the ACL 
had the largest proportion of significant items and the CPI the 
smallest proportion. The OBI scales had lower internal consistency 
than the scales from the other three item pools. These data are not 
sufficient evidence to establish the superiority of one item pool 
over another. The validity coefficients of scales composed from 
the various item pools are the most important consideration, and 
these are discussed later. 


Item Content 


The item content of the scales gives an indication of the kind of 
person who is likely to achieve; although it is important to re- 
member that the items are self reports and that they were 
identified by group comparisons so that all items do not necessarily 
apply to all students. With these qualifications in mind the items 
significantly related to each criterion were grouped into what 
seemed to be homogenous content categories as follows: 

Content of the grade scales. The high grade group, when con- 
trasted with the low grade group, more frequently endorsed items 
suggesting that they are religious and involved in church activities; 
are interested in school and value school work highly; are interested 
in music and participate in musical activities; are hard workers 
with well established work habits; and are shy, socially with- 
drawn, and introverted. The low grade group on the other hand 
more frequently endorsed items suggesting that they are some- 
what mischievous and thrill-seeking; are likely to engage in a long 
list of activities usually considered “fun” for young people; are 
erratic and undependable; and are interested in occupations which 
might be considered exciting or dangerous. 


na SNES Sati 
2The items and scoring keys for the scales developed in this study are 
available to responsible persons from the author on request. 
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The following adjectives were selected as representative of the 
ACL items significantly differentiating the high and low grade 
groups: High grade students described themselves as ambitious, 
capable, conscientious, dependable, efficient, helpful, methodical, 
modest, patient, quiet, resourceful, self-confident, timid, well- 
adjusted, and withdrawn. Low grade students described themselves 
as boastful, carefree, careless, cynical, disorderly, high-strung, im- 
pulsive, irresponsible, lazy, messy, rebellious, and sophisticated. 

A few individual CPI items deserve mention because they rein- 
force the general impression from the foregoing discussion that the 
student who gets good grades is likely to be compulsive and con- 
forming. For example, the high grade group more frequently re- 
sponded “True” to the following items: “I am stricter about right 
and wrong than most people.” “I would disapprove of anyone’s 
drinking to the point of intoxication at a party.” “I keep out of 
trouble at all costs.” “I consider a matter from every standpoint 
before making a decision.” “I always like to keep my things neat 
and tidy and in good order.” 

Content of the extracurricular achievement scales. The group of 
achievers, when contrasted with the nonachievers, more frequently 
endorsed items suggesting that they engage in a variety of extracur- 
ricular activities including music, speech, drama, science, art, etc. 
(all activities similar to the criterion achievements) ; are religious 
and involved in church activities; are outgoing and dominant in 
interpersonal situations; date frequently; are ambitious and hard 
working; and are interested in a variety of artistic and intellectual 
occupations. The nonachievers on the other hand more frequently 
endorsed items suggesting that they have a low energy level and are 
overly sensitive to the opinions of others. 

The following adjectives were selected as representative of the 
ACL items significantly differentiating the achieving and non- 
achieving groups: Achievers described themselves as aggressive, 
alert, ambitious, artistic, attractive, clever, confident, cooperative, 
deliberate, dominant, egotistical, energetic, generous, helpful, in- 
dependent, ingenious, mature, original, persistent, responsible, so- 
phisticated, unconventional, versatile, well thought of, and witty. 
Nonachievers described themselves as lazy, quiet, shy, slow, and un- 
ambitious. 
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Cross-validation 


The intercorrelations of the various scales, test scores, and high 
school rank (HSR) are shown for the Merit cross-validation 
group in Table 2 and for the normative group in Table 3. 

The correlations shown in these two tables lead to the following 
conclusions: 

(a) The two criteria had low positive correlations with each 
other for both sexes and both samples. These correlations are low 
enough for the grade and extracurricular criteria to be considered 
as independent for all practical purposes. 

(b) Among the nonintellective predictors, the CPI scale was the 
best predictor of grades (except for Merit girls) and the OBI scale 
was the best predictor of extracurricular achievement. For both 
criteria the CPI and OBI scales were substantially better predic- 
tors (average validity .23) than were the ACL and VIP scales 
(average validity .11). 

(c) The two criteria were differentially predictable by the non- 
intellective scales. In only five of 32 instances did a scale fail to 
correlate higher with the criterion for which it was derived than it 
did with the other criterion, and none of these five instances in- 
volved the CPI or OBI. The scales for the two criteria were nega- 
tively correlated in the OBI and VIP and positively correlated in 
the CPI and ACL. 

i (d) In the Merit sample the NMSQT and SAT had slight predic- 
tive validity for both criteria for boys and no validity for either 
criterion for girls; however, low correlations are to be expected in 
this instance because of restriction of range of test scores in the 
Merit sample. In the normative sample the NMSQT had some 
validity for both criteria for boys but was related only to the 
grade criterion for girls. 

(e) HSR was related to the grade criterion for both sexes and 
both samples, but not to the extracurricular achievement criterion. 
It has been suggested that rank in high school class uncorrected for 
class size is as effective a predictor of college grades as is percen- 
tile rank. This was true for boys in both samples, but not for girls. 

(£) Two reports of high school rank were available for the Merit 
sample, one from the school and one from the student. The correla- 
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tion between these two reports was 81 for boys and .82 for girls. 
This is a somewhat lower validity for self reported grades than 
others have reported. A possible reason is that in this study students 
‘were asked to report both rank and class size, from which percen- 
tile rank was computed. This task may have been more difficult than 
the usual report of a grade average. The restricted range of class 
ranks among the Merit students is another possible reason, What 
ever the source of error in the self reported HSR may be, it is in- 
deed a peculiar kind of error: the correlation with college grades 
for girls is the same for self reported HSR as for school reported 
HSR, but for boys self reported HSR has a substantially, but not 
significantly, higher correlation with college grades than does school 
reported HSR. 

(g) It is possible to form a rough ordering of the three classes of 
predictors (test scores, non-intellective scales, and HSR) in terms 
of their validity for predicting the two criteria, and this order holds 
in general for both samples and both sexes. To give an indication of 
the relative size of the coefficients the average validity of the OBI 
and CPI scales will be used to represent the non-intellective scales, 
the NMSQT Composite to represent the aptitude tests, and percen- 
tile rank to represent HSR. For predicting college grades, high 
school grades were the best predictor (average validity .33), fol- 
lowed by the nonintellective scales (average validity .27), and 
finally by the aptitude test (average validity .12). For predicting 
extracurricular achievement the nonintellective scales were the best, 
predictor (average validity .19), followed by high school grades 
(average validity .10), and the aptitude test (average validity 
.07). Grades were predicted with higher validities by all three 
classes of predictors than was extracurricular achievement. 

(h) Girls tended to achieve at a higher level than boys in most 
respects. They made better grades in high school and college; they 
obtained higher scores in both groups on nonintellective scales; and 
they obtained higher verbal test scores. There was no sex difference 
in extracurricular achievement, and boys obtained higher scores on 
quantitative tests. The achievement of boys was somewhat more 
predictable than that of girls: the nonintellective scales had higher 
validities for boys and in the Merit sample the test scores also had 
higher validities for boys. 
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Multiple Correlations 


A number of multiple regression equations were computed for pre- 
dicting each criterion from various combinations of predictors in the 
various samples, Since these equations showed little consistency 
from one sample to another and since the multiple correlation co- 
efficients are subject to an unknown amount of shrinkage, they will 
not be reported in detail here. However, a few general patterns 
emerged consistently from the various analyses: (a) The nonintel- 
lective scales derived from the four item pools did not contribute 
unique information. Once the scale with the highest correlation with 
the criterion was taken into account, the partial correlations of the 
other scales with the criterion were nonsignificant. (b) The non- 
intellective scales added significantly in most instances to the pre- 
diction of the grade criterion that was possible with HSR and 
the SAT or NMSQT Composite. (c) HSR and the test scores in 
most instances did not add significantly to the prediction of extra- 
curricular achievement that was possible with a single nonintellec- 
tive scale. 

Discussion 

The scales to predict grades developed from the OBI and CPI 
item pools add significantly to the prediction of college grades that is 
possible from test scores and high school rank. The extracurricular 
achievement scales seem to be the best available predictors of 
extracurricular achievement. Moreover, these scales appear to be 
particularly robust: developed on a sample of very able students 
of both sexes, they maintained their validity when applied to boys 
and girls separately and when applied to a sample of students 
considerably lower on the ability scale than the sample on which 
they were derived. 

These results warrant tentative use and further trial of these 
scales in actual selection situations.’ If the selection context does 
not greatly affect their validity, the scales will provide an increment 
in the accuracy of prediction of college performance. 
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Extracurricular achievement in this study was not as predictable 
as were grades. Moreover, the predictors of extracurricular achieve- 
ment were primarily high school activities and extraversion—traits 
which may not be considered valuable in themselves. This raises 
the question of what weight should be given in college and scholar- 
ship selection programs to predictors of extracurricular achieve- 
ment. It is conceivable that some admissions officers may wish to 
select potential extracurricular achievers to promote campus activi- 
ties or to balance an overly studious student body. However, in 
most selection programs the justification for selecting potential 
extracurricular achievers,must be based on the assumption that 
such achievers will make socially valuable achievements after col- 
lege. Although this assumption seems reasonable enough, there is 
little good evidence either to support or to refute it. 

There is a clear need for better criteria of success in college, 
and the lack of good criteria is one of the main difficulties in improv- 
ing predictors. Since the criterion value of any index of college 
achievement depends in large part on its relationship to post col- 
lege achievement, studies of the correlation of college behavior with 
socially significant achievement after college might help to clarify 
the criterion problem. 

A specific finding which deserves comment is the low predictive 
validity of the aptitude tests in this study. This might be expected in 
the Merit sample with its restricted range of scores, but it was also 
true in the normative sample. The correlations between NMSQT 
Composite and college grades in the normative sample of .18 for 
boys and .23 for girls are considerably below the correlations in 
the .50’s reported in the NMSQT Manual (Science Research Asso- 
ciates, 1964) for students attending particular colleges. This 
shrinkage in validity is probably due to the fact that the students 
in the present samples attended many different colleges, with the 
high scoring students attending colleges where the academic 
competition is greater than for the low scoring students. The shrink- 
age in validity for the test scores seems greater than for HSR 
(normative sample validities of .34 for boys and .38 for girls). The 
great differentiation of colleges on tested ability may effectively at- 
tenuate the validity of test scores for predicting college grades in a 
heterogenous group of colleges, leaving the nonintellective factors, 
on which colleges are not so differentiated, as the best predictors. 
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This reasoning would seem to indicate that those who deal with 
students attending a variety of colleges should develop ways of 
taking the college into account if they wish to use grades as an in- 
dex of success. 

Validity is often not the sole value in a selection program, and the ~ 
use of nonintellective scales raises some issues which, although al- | 
ways present, are hidden when intellectual predictors are used. ; 
Stalnaker (1965) stated the issue well when he said (propheti- 
cally), “In a program very much in the public eye, predictive 
validity alone cannot rule . . . Suppose there should develop sound © 
evidence that among the highly intelligent, the most conforming, a 
compulsive, dependent, unoriginal individuals do best in college,” 
Should we then try to limit our selection to students having thesé 
characteristics” (p. 135)? Stalnaker’s hypothetical example is only 
a slight exaggeration of the content of the nonintellective grade 
scales. The extracurricular scales on the surface would seem to select 
more the “All American Boy” type, but should a student be award 
a scholarship because of his broad interests and frequent dates? 

The personality traits of the selected students become explicit 
with the use of nonintellective predictors, but selection for person= A 
ality characteristics is implicit in all selection programs. For exam- " 
ple, unpublished comparisons of the Merit and normative samples 
used in this study reveal that the NMSQT tends to select a socially — 
withdrawn, studious, introversive character. Since the composition — 
of social groups in our society is increasingly determined by cen- 4 
tralized selection programs, more attention should be given to the 
type of person identified by the various selection strategies. The ex- 
plicit recognition of the role of personality traits that is inherent 
in the use of nonintellective predictors may help focus attention ~ 
on this problem. 


Summary 


Scales for predicting first year college grades and extracurricular 
achievement were developed by item analysis from each of four item 
pools (the California Psychological Inventory, CPI; the Voca- 
tional Preference Inventory, VPI; and Adjective Check List, ACL; 
and an experimental Objective Behavior Inventory, OBI) using a 
sample of 1013 National Merit Finalists. The scales were cross- 
validated using samples of 179 male and 138 female Merit Final- 
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ists and 201 male and 218 female students of average ability. The 
CPI and OBI scales had higher validities than those developed 
from the ACL and VPI. The best predictor of college grades was 
rank in high school class (HSR) followed by the nonintellective 
grade scales and finally by aptitude test scores. The nonintellective 
scales added to the prediction of grades in a regression equation 
including HSR and test scores. The best predictors of extracurricu- 
lar achievement were the prediction of extracurricular achieve- 
ment in a regression equation including the nonintellective scales. 
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THE DIFFERENTIAL PREDICTION OF COLLEGE GRADES 
FROM BIOGRAPHIC INFORMATION 


PATRICIA W. LUNNEBORG anv CLIFFORD E. LUNNEBORG 
University of Washington 


A present-day concern of researchers into higher education is 
prediction of academic accomplishment from nonacademic vari- 
ables, Holland (1961) was interested in establishing a new basis for 
awarding scholarships to high aptitude students through the use of 
nonintellectual indices of creative performance. Other investigators 
such as Astin (1964) and Heilbrun (1965) have focused upon the 
dropout problem in higher education and the personal and environ- 
mental factors associated with failure. French (1963) wanted to 
compare aptitude, interest, and personality measures as differen- 
tial predictors of success in maj or-field areas as a step in developing 
a multi-factor battery for use in counseling. 

This study was both part of the current trend and apart from it in 
two important respects. Like French’s work, it was concerned with 
differential prediction in course areas; however, the improvement 
over intellective predictors stemmed not from more testing—of 
needs, attitudes, personality traits. The goal was rather to identify 
useful biographic items from data already at hand in admissions or 
application forms. These forms remain the same despite the fact 
that student selection is now based primarily on two kinds of in- 
formation, overall high school GPA and aptitude and achievement 
test scores. To be able to use background information would save 
valuable hours of testing and restore purpose to the time and care 
students take to complete applications in the first place. 

The second unique goal is bound up with the philosophic rationale 
for differential as opposed to absolute prediction. Differential pre- 
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diction is based on the assumption that most decisions are made be- 
tween one course of action or study and another, and that the 
most useful data to individuals facing such choices are predictions 
based on differences in performance between areas, rather than pre- 
dictions based upon what is common among criterion areas. Maxi- 
mum utility to the individual is placed above maximizing predic- 
tive efficiency for the group. In line with making a differential 
prediction program as comprehensive as possible, biographic vari- 
ables would serve to expand the factorial composition of an aca- 
demic battery and, in turn, improve prediction in areas of divergent 
thinking such as art, architecture, and music, as well as facilitate 
adding technical and business training criteria for community col- 
leges. 


Method 
Subjects 


The sample consisted of the 520 freshmen (295 males and 225 
females) at the University of Washington who took the Washing- 
ton Pre-College (WPC) Test Battery (1965) when they entered 
college in fall quarter 1965. Inasmuch as almost all students take 
this state-wide Battery as high school seniors, the present sample 
was very atypical in this respect, largely due to the fact that 63 
per cent were nonresidents compared with only 9 per cent nonresi- 
dents in the total freshman class. The sample also included a com- 
paratively large number (12%) who entered with transfer credit 
(less than 45 hours). However, all were U. S. citizens, and were 
born in or after 1940. 


Predictors 


The predictors consisted of age, sex, and (1) six cumulative high 
school GPA’s: English, foreign languages, mathematics, natural 
sciences, social studies, and full-credit electives, (2) twelve Battery 
test Scores: vocabulary, English usage, spelling, reading speed, 
reading comprehension, quantitative skills (3 parts), applied mathe- 
matics, mathematics achievement, spatial ability, and mechanical 
reasoning, (3) 71 items from the admissions application to Wash- 
ington higher institutions, and (4) 62 items from a “Survey of Col- 
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lege Plans,” a half-hour experimental questionnaire administered 
with the Battery. Anastasi’s (1960) Biographical Inventory items 
were used as a guide in developing the scoring keys for the latter 
two predictor sources. Questionnaire items included some from 
Anastasi’s Fordham inventory and from Peterson’s survey (1964). 
Criteria 

The WPC Testing Program provides every student entering a 
Washington college with grade predictions for 42 college subject 
areas and for the all-college GPA based upon the Battery, the six 
high school GPA’s, age, and sex. For the present sample the follow- 
ing GPA’s based on one quarter of university work served as cri- 
teria: over all subjects, English composition, mathematics, foreign 
language, and physical science (chemistry and physics). Other 
criteria were, change in major at the quarter’s end and withdrawal 
from school, 


Procedure 


Predictor intercorrelations and predictor-criterion correlations 
were computed, and served, together with observed distributions 
of Ss responding on dichotomized biographic items, as the basis for 
choosing 32 biographic variables to be studied further, together with 
the other (20) predictors. 

Sequential predictor selection analyses on these 52 variables were 
conducted for each of the seven criteria using the iterative predictor 
selection technique of Horst (Horst and Smith, 1950), No limit 
was set on the number of variables to be included in each best set 
so that a greater number of potentially useful variables would be 
identified. The purpose of these selections was, however, to re- 
strict further the pool of predictors for differential prediction and to 
insure that this pool included variables which in combination were 
related to the course areas. 

The final pool consisted of the 19 predictors which were among 
the first six selected above for each of the four course areas. The 19 
selected variables were then examined for contribution to differen- 
tial prediction by the Horst technique (1954), and corrected mul- 
tiple correlations determined for the eight best predictors with the 
four course grades and overall GPA. 
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_ Results 


Table 1 presents the results of the sequential predictor selection — 
of the 52 variables in terms of order of selection and standard — 
partial regression weights. Multiple correlations (R.) given here 
and throughout this report are corrected coefficients, i.e., reduced to 
reflect expected between-sample shrinkage due to sample size and 
number of predictors as prescribed by Snedecor (1946, p. 348). 

The most striking aspect to the selections was the frequency with 
which biographic variables appeared. Looking across the seven 
criteria at the first six predictors selected, only one high school 
grade average may be found and only five test scores, in contrast 
to twenty-two of the 32 biographic variables. The multiple correla- 
tions for the five GPA criteria were substantial in size and not un= 
like those found when these criteria are more reliable and long-term. — 
When the criteria were nonintellective, withdrawal or change in ma- 
jor, the intellective predictors disappeared entirely from the scene. o 

Of the 32 biographic variables entered into sequential predictor 
selection, only five were never selected. Although living on campus 
as opposed to commuting was positively correlated with grades in 


TABLE 1 


Order of Selection of Predictors and Standard Partial Regression Weights 
for Seven Criteria of Freshman First Quarter Performance 


(N = 520) 
Criterion: Beta weight Criterion: Beta weight 
Freshman English GPA Freshman Mathematics GPA 
(18 predictors selected) (9 predictors selected) 
English usage test 38 Mathematics achievement test 37 
Philosophy of higher education: Age 30 
Vocational (Q) -13 Father college graduate (A) 21 


Level of intended vocation (A) —13 Level of intended vocation(A) 19 
1 


Hours of high school study (Q) 12 Later-born (A) 6 
Reading comprehension test 18 Expects literary activity in 
Major: Humanities (A) 14 college (Q) 19 
Quantitative skills test, Part B Church member (A) Ww 
(Quantitative Judgment) 15 Father’s occupation: Business- 
High scnoel cae govern- organizational (A) -12 
ment activity 10 Hours of high school stud; 11 
Expects literary activity in Be = 42 pth 
oe (Q) 12 Freshman Physical Science GPA 
ony om (A) 09 (16 predictors selected) 
other deceased (A) —08 Intended vocation: Technical 
Father’s occupation: Business- (A) —54 
organizational (A) —07 Quantitative skills test, Part B 
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oo oom 


Criterion: Beta Weight Criterion: Beta Weight 
Age 08 (Quantitative Judgment) 41 
R. = 67 Age 40 
Freshman Foreign Language GPA Father’s occupational level (A)—23 
_(10 predictors selected) Father's occupation: Business- 
English usage test 29 organizational (A) -ll 
Hours of college study (Q) 21 High school student govern- 
Intended vocation: Technical ment activity (A) 23 
(A) —29 Philosophy of higher education: 
Mathematics achievement test 33 Vocational (Q) 22 
Father's occupation: Hours of high school study (Q) 17 
Technical (A) 17 Church member (A) -14 
High school foreign language English usage test 22 
GPA 14 High school academic honors 
Spatial ability test —17 (A) = 
Mother college graduate (Q)—15 High school mathematics GPA 28 
Later-born (A) 12 Intended vocation: 
Long-standing college plans Verbal-cultural (A) 15 
(Q) 11 Father’s occupation: 

R. = .54 Technical (A) 19 
Freshman Cumulative GPA Level of intended vocation (A)—14 
(11 predictors selected) College motive: to learn 

Quantitative skills test, Part B certain subjects (Q) -14 
(Quantitative Judgment) 44 R. = .73 
High school favorite: Foreign Withdrawal 
language (Q) 11 (5 predictors selected) 
Vocabulary test 10 Mother deceased (A) ; 20 
Intended vocation: Technical —27 Philosophy of higher education: 
32 Nonconformist (Q) 24 


Age 

Long-standing college plans y 
(Q) 

High school mathematics GPA 17 

Hours of high school study (Q) 09 

Washington State home town 


(A) -10 
High school student govern- 
ment activity (A) 
Mother spore (A) 07 
oe = a 


High school academic honors 

(A) —16 
Hours of college study (Q) —11 
Father's occupational level (A) —10 

R, = 35 
Change of Major 
(8 predictors selected) 

Philosophy of higher education: 

Nonconformist (Q) 28 
Church member (A) -18 
Father college graduate (A) 19 
Cultural interest scale 


(16-point) score (Q) -14 
Intended vocation: Verbal- 

cultural (A) 13 
Father’s occupation: Business- 

organizational (A) 10 
Level of intended vocation (A) —11 
Washington State home town 

(A) 

R. = .35 


eens pS SENETA 


Note,—Decimal points’ omitted. 
representing the most prestigious. 


A indicates application blank item; Q indicates questionnaire 


item. Levels according to Roe (1956) with 6 representing the lowest level of occupation and 1 
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concurrence with Astin’s findings (1964), it was highly correlated 
with level of father’s occupation and mathematics achievement, and 
thus went unselected. Number of high school honors and a scholar- 
ship based on scholarship were not selected because of high cor- 
relations with high school GPA’s and test scores. The other two 
predictors passed over were preference for an academic or profes- 
sional life, and having an intellectual philosophy of higher educa- 
tion (65 per cent of the sample). Three predictors overlapping be- 
tween application and questionnaire were only selected from one or 
the other source, 

Only two of Roe’s (1956) occupational types were included in the 
sequential selection analyses, business-organizational and technical, 
because the other six types each held only 4 per cent to 10 per 
cent of the fathers. With a larger sample, the less populated occupa- 
tions might be expected to contribute to prediction as well, e.g., 
physical science grades correlated 21 with science as father’s oc- 
cupation. This holds true as well for students’ intended occupa- 
tions of which only the technical and verbal-cultural types (law, 
teaching, journalism) were popular enough to be studied with this 
sample. 

The results of the selection of variables for their contribution to 
differential prediction are summarized in Table 2. This selection 
model (Horst, 1954) insured that the first selected variable, in this 
instance the English usage test, was the one which best predicted 
differences among the four criteria at hand. The second selected 
variable, intent to engage in a technical vocation, was that vari- 
able among the eighteen remaining which best extended this dif- 
ferential prediction. Successive additions were designed to include, 
at each stage, that variable which best complemented the predictors 
already selected. Table 2 presents only the first eight selected to- 
gether with their regression weights for corrected multiple correla- 
tions for the overall first quarter GPA and freshman course areas. 
These weight determinations, utilizing once again the sequential 
technique (Horst and Smith, 1950) assigned optimal weights to 
each of the eight predictors. 


y The present study offers challenging support for the notion that 
iographic information of the kind easily available from admissions 
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TABLE 2 


3 Order of Selection and Standard Partial Regression Weights for Predictors Selected 
for Contribution to Differential Prediction Efficiency (Decimal points omitted) 


College GPA criteria 


Predictors in order Mathe- Foreign Physical 

of selection (7) A: $i English matics language science Overall 
English usage test 12665 12665 50 —00 25 15 20 
Intended vocation: 

Technical 05270 17935 —17 -17 —30 —60 —29 
Philosophy of higher 

education: 

Vocational 05535 23470 —13 —04 —08 20 —09 
Level of intended 

vocation 04817 28281 —10 20 11 —08 00 


a High school 
student govern- 


ment activity 04751 33032 10 -07 —08 21 08 
Age 03773 36805 10 26 09 34 23 
Hours of college 

study 03429 40234 09 —00 21 —02 06 
Mathematics 

achievement test 02911 43145 19 44 35 44 42 

Re 63 24 46 58 49 


sa eee ae E a E E ERE EN E E 

Note.—¢: is Horst’s (1954) index of differential prediction efficiency for the first ¢ variables selected, At each 
stage that predictor is selected which will make the greatest contribution (A1) to the index. 4 is proportional to 
the average of the variances of the predicted criteria minus the average of the covariances among the predicted 


criteria. 


| applications or brief questionnaires can effectively contribute to 
| the prediction of academic performance. For each of seven criteria 
| of freshman performance, a pool of 32 biographic variables con- 
f tributed a greater proportion of predictors compared with intellec- 
| tive variables. Then, of the 19 predictors among the first six selected 
| for the four course criteria, only five were of the pre-college, intel- 
| lective variety—high school foreign language GPA, a verbal and 
a quantitative aptitude test, and a verbal and a quantitative 
achievement test. Lastly, that one or more biographic variables con- 
tributed to differential prediction is not surprising; indeed, it was 
in anticipation of factorial dissimilarity that they were explored. 
But the selection of so many nonintellective variables prior to some- 
thing as basic to college grades as quantitative skills or achieve- 
ment was not expected. Subject to replication, it appears that non- 
intellective variables of these types possess great potential for the 
expansion of the differential battery because of their factorial 
complexity; not only were they different from intellective factors 
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already present, but they were different from one another. Nonin- 
tellective contribution to prediction of courses less like the p 
college variables than English composition, physical science, a 
foreign language courses should be even greater than for these three 
which are among the six most predictable from the established 
WPC Battery (1965). f 

Some discussion is in order concerning the consistency of thi 
relationships between nonintellective variables and academic ¢ 
teria identified in this study with those already reported. TI 
major area of agreement with Astin’s (1964) dropout study was 
contribution to college success of socioeconomic level as meas 
by father’s and mother’s education, and father’s occupational type 
and level. Even birth order was a useful predictor in the present 
study, in contrast to the lack of relationship between demographi 
predictors to grades when the range of talent is restricted (Holland, 
1961). Several items which Anastasi (1960) found associated with 
college success were also confirmed: hours of high school study, vo- 
cational goal (Teaching or, in this instance, “verbal-cultural”), — 
high school student government activity, expectation of literary 
activity in college, language as favorite high school subject, and 
academic honor(s) in high school. Despite the fact that 76 per cent 
of the present sample had participated in three or more high school — 
activities, and that 64 per cent anticipated this same level of extra- 
curricular involvement in college, activities were not predictive, 
while they were in the Fordham study. Two other useful items at 
Fordham that surprisingly showed no relationship to grades in the 
present investigation were plans for post-baccalaureate study and 
anticipation of difficulties concentrating or studying. Study difi- — 
culty was correlated solely with withdrawal. 

Perhaps the first investigation of biographical factors and aca- 
demic achievement, Myers’ (1952) study at an eastern women’s — 
liberal arts college identified college success with the Jewish religion, 
urban living, foreign born parents, and number of high school offices 
held, none of which bore any relationship to freshman grades at 
the University of Washington. These findings, together with the 
lack of validity for age, father’s occupation, and parents’ educa- 
tion in Myers’ study, recall the substantial restrictions on the 
Present sample which limit any generalizing about the effects of so- 
cial class or study habits or educational philosophy on college 
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achievement. Aside from the real differences between the popula- 
tion sampled in these two studies, the relatively large number of 
variables investigated here in relation to the number of subjects 
means that in subsequent samples not only will the weights in multi- 
ple prediction change, but the contributing correlation coefficients 
themselves will vary. The importance of the present study lies 
rather in the confirmation provided French’s (1963) conclusion 
that while intellective measures contribute most to the absolute 
prediction of college grades, certain nonintellective measures con- 
tribute most to differential prediction and thus have a place in 
prediction when the goal is maximum utility to the individual. 


REFERENCES 


Anastasi, Anne, The Validation of a Biographical Inventory as a 
Predictor of College Success. College Entrance Examination 
Board Research Monograph, No. 1. New York: College Entrance 
Examination Board, 1960. 

Astin, A. W. Personal and Environmental Factors Associated with 
College Dropouts among High Aptitude Students. Journal of Edu- 
cational Psychology, 1964, 55, 219-227. 

French, J. W. Comparative Prediction of College Major-Field 
Grades by Pure-factor Aptitude, Interest, and Personality 
Measures, EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1963, 
23, 767-774. 

Heilbrun, A. B. Personality Factors in College Dropout. Journal of 
Applied Psychology, 1965, 49, 1-7. 

Holland, J. L. Creative and Academic Performance among Talented 
‘Adolescents. Journal of Educational Psychology, 1961, 52, 136- 


147. 

Horst, P. A Technique for the Development of a Differential Pre- 
diction Battery. Psychological Monographs, 1954, 68, No. 9, 
Whole No. 380. A 

Horst, P. and Smith S. The Discrimination of Two Racial Samples. 
Psychometrika, 1950, 15, 271-289. k 

Myers, R. C. Biographical Factors and Academic Achievement: 
‘An Experimental Investigation, EDUCATIONAL AND PSYCHOLOGICAL 
MEASUREMENT, 1952, 12, 415-426. 

Peterson, R. E. Some Biographical and Attitudinal Characteristics 
of Entering College Freshmen. ETS Research Bulletin, Educa- 
tional Testing Service, Princeton, New Jersey, 1964. 

Roe, Anne. The Psychology of Occupations, New York: John 

iley & Sons, 1956. 

sanders G. W. Statistical Methods, (4th ed.) Ames, Iowa: Iowa 
State College Press, 1946. 

Washington Pre-College Testing Program. Counselor's Manual 
1965-1966. Seattle: Washington Pre-College Testing Program, 


1965. 


EDUCATIONAL AND PSYCHOLOGICAL MEA: 
1966, 26, 927-933, g aik 


VALIDITY OF SOME OBJECTIVE SCALES OF 
MOTIVATION FOR PREDICTING ACADEMIC 
ACHIEVEMENT! 


EDWARD J. FURST 
University of Arkansas 


Tum main purpose of this study was to test the hypothesis that a 
self-rating scale, designed to yield a fairly direct measure of aca- 
demic achievement-motivation, will show higher correlations with 
achievement than will a measure of a person’s generalized need for 
achievement. This hypothesis was suggested by Myers (1965) but 
his study did not present a test of it with data on the same in- 
dividuals. 

Other purposes were to check on the comparative predictive va- 
lidity of (1) similar scales such as those derived from the Stern 
Activities Index (SAI), (2) a simple preference between academic 
and nonacademic subjects (Mayhew, 1965), (3) conventional 
verbal and nonverbal measures of general ability, and (4) selected 
combinations of these variables. 


Description of Predictor Variables 


The following were the predictor variables used: 
1. Total score on Achievement Motivation scale 
2. Achievement scale (Ach), SAI 
3. Counteraction scale (Ctr), SAT 
4, (Ach + Ctr), SAI 


1 This study represents a part of Project HS-082 supported by the Coopera- 
tive Research Program of the Office of Education, U. 8. Department of Health. 
Education, and Welfare, and conducted at the Ohio State University. 

Special thanks are due Dr. ‘Albert E. Myers for permission to adapt his 
version of an achievement motivation scale developed at Educational Test- 
ing Service, and Lee C. Lee for help with data processing. 
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5, Humanities-Social Sciences scale, (Hum), SAT 

6. Understanding scale (Und), SAI 

7. (Hum + Und), SAT 

8. Like academic subjects vs. like nonacademic subjects; scored 

1,0 

9. Score on verbal measure of general ability 
10. Score on nonverbal measure of general ability 

The Achievement Motivation scale was an adaptation of one 
used by Myers (1965). It consisted of the following nine statements 
to be rated on a five-point scale: 5-very true of self; 4-fairly true; 
3-a little true; 2-hardly true; 1-not at all true. 

1. When you know there are going to be one or two questions on 
an examination from outside reading assignments, you always 
read all the material. 

2. You regard yourself as a more consistent and harder worker 
in your classroom assignments than the typical student in your 
classes. 

8. Others (not your good friends) have thought of you as one 
who “missed some of the fun” because you were so serious. 

4. You think your fellow students in school think of you as & 
hard worker. 

5. Most of your teachers probably think of you as one of their 
hardest workers even though not necessarily one of the smart- 
est. 

6. Other interests (sports, extracurricular activities, or hobbies) 
prevent you from obtaining an excellent rating or mark for 
effort in school work. 

7. You have a very strong desire to excel academically. 

8. You try harder to get on the school honor roll or merit list 

“than the average student in your class. 

9. You try to do most jobs at least a little better than what you 
think is expected. 

Item 6 was scored in reverse. The possible range of scores was from 
9 to 45. Retest reliability for a one-month interval was .77 (V = 
52). This scale, the simple preference, and the SAI were given 
early in September, 1965. 

Measures of ability differed from sample to sample, depending 
upon the school’s testing program. For the samples as numbered 
here, the respective verbal measures were: I—California Language 
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Test, Form X, 1957 Revision, Fundamentals; IJ—Differential 
Aptitude Tests (DAT), Verbal Reasoning; I11I—California Test of 
Mental Maturity (CTMM), 1957 S-Form, Language; IV—Lorge- 
Thorndike Intelligence Tests, (LTIT) Verbal; V—CTMM, 1963 
§-Form, Language. The respective nonverbal measures were: I— 
none available; II—DAT, Numerical Ability; III—CTMM, Non- 
Language; IV—LTIT, Non-Verbal; V—CTMM, Non-Language. 
Generally, these tests had been given between the middle of the 
eighth grade and the middle of the ninth. 


Samples 


The samples consisted of ten classes beginning the ninth grade 
in September, 1965 and taking a one-semester trial course in eco- 
nomics. Classes taught by the same teacher, and by teachers in the 
same or closely similar schools and communities, were combined so 
as to yield groups with larger N’s. Each of the five groups consisted 
of two such classes. The groups varied in ability, as estimated from 
tables of norms giving centile equivalents for average verbal scores, 
Groups I and IV were about average—60th and 55th, respectively; 
III was somewhat above average—76th; II and V were much 
above average—92nd and g5th—and were largely college-ori- 
ented samples from well-to-do suburbs. 


Criterion Measures 


Final course marks for the two semesters of the eighth grade and 
the first semester of the ninth comprised the criteria of achievement. 
So as to make records comparable from sample to sample, only 
marks in English, social studies, mathematics, and science were 
used, Letter marks were converted to the usual 0-4 equivalents. 
Inasmuch as the full eighth-grade record correlated about .8.with 
the ninth-grade GPA, and inasmuch as these respective GPA’s 
showed a highly similar pattern of correlations with the predictors, 
the total eighth and ninth-grade GPA was used as the single 


criterion. 
Results 
Table 1 is of interest for any evidence of differences between 


th the exception of the Hum scale, boys and girls 


boys and girls. Wi ! 
did not differ significantly on any of the variables. It was therefore 
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decided to leave each sample as it was rather than to conduct 
further studies of boys and girls separately. 

Table 2 shows zero-order correlations for the five samples. On 
the whole (taking the median values), the results confirm the 
hypothesis that an objective, fairly direct measure of desire to 
achieve in school is a better predictor of academic achievement 
than is a measure of generalized need for achievement (Ach or 
Ach + Ctr), or measures of intellectual interests in general (Hum 
or Und, or Hum + Und, or liking for academic subjects). These 
correlations agree closely with the corresponding ones in Table 1 for 
all boys and all girls, respectively. The median correlation of .50 
between the Achievement Motivation Scale and GPA is the same 
as the figure reported by Myers (1965) for his samples of eleventh- 
grade, college-oriented, students. 

The combinations of variables in Table 3 are limited in number 
because they were chosen so as to bring out certain critical com- 
parisons, rather than to determine the best combination or to study 


the incremental validity of each predictor. A comparison of the 


TABLE 2 


Correlations between Predictors and Criterion; M ‘eans and 
Standard Deviations for Measures of Ability* 
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* Significant at the .05 level. 

** Significant at Re 
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TABLE 3 
Multiple Correlations of Certain Combinations of 
Predictors and the Criterion 
Group 
I Il m IV v 
Predictors (N = 41) (N = 41) (N = 49) (N = 37) (N = 60) 
1,4 R .58 44 -76 23 54 
F 9.42** 4.62* 31.26** -96 11.83** 
af. 2,38 2,38 2,46 2,34 2,57 
7,8 R 31 14 51 24 53 
F 2.07 -39 7.88** 1.08 11.31** 
d.f. 2,38 2,38 2,46 2,34 2,57 
1, 4, 7,8 R 61 S TT .33 .65 
F 5.45** 2.42 15.97** -98 9.94** 
df. 4,36 4,36 4,44 4,32 4,55 
1, 4,7,8 
9, 1 R -86 76 -80 66 81 
F 19.34** 7.53** — 12.69** 3.82 17:07% 
d.f. 5,35 6,34 6,42 6,30 6,53 


* Significant at the .05 level. 
** Significant at the .01 level. 


first row of R’s with the first row of r’s in Table 2 shows that the 
composite score, Ach + Ctr, adds little, if any, to the predictive 
power of the Achievement Motivation scale. Further, the substi- 
tution of variables 7, Hum + Und, and 8, liking academic sub- 
jects, for variables 1 and 4, results in considerably less predictive 
power in three of the five samples, This comparison is of importance 
because it pits measures of sheer desire to achieve against measures 
of interest in particular intellectual activities. (In factorial analy- 
ses of the SAI scales by the writer, these respective pairs of scales 
came out as independent factors for both the boys and the girls, 
N’s = 229 and 165, respectively.) The third row of R’s in Table 3 
gives an approximation to the probable upper limit of predictive 
power of the several motivational and interest variables taken as a 
weighted composite, It may also be noted that the increment over 
the composite of 1 and 4 was slight in four of the five samples. The 
last row of R’s shows that the addition of one or two measures of 
general ability to the four-variable motivational-interest com- 
posite increased R substantially except in the one sample where 
R was already on the order of .8, Thus, the motivational-interest 
composite by itself is not enough for best prediction. 

Finally, the data in Table 4 indicate that achievement motiva- 
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TABLE 4 
Correlations of Scores on Achievement Motivation Scale 
and Measures of General Ability 
Group 
Measure of General I II Til IV v 
Ability (N=41) (V=41) (N=49) (N=37) (N =00) 
Verbal or Language .32* —.12 .38* .34* .32* 
Non-V. or Non-L. — .09 .26 .13 .13 


* Significant at the .05 level. 
tion is largely independent of general ability, particularly the NV 
or NL measures, in groups with this level and range of ability. 

Discussion 

Tt has been shown that a simple, objective, and fairly direct self- 
rating scale of motivation to do well in school tends to give better 
predictions than more generalized measures of need to achieve or 
measures of intrinsic intellectual interests. The scale essentially 
samples aspects of the self-concept, reflecting both the person as he 
sees himself and as he thinks his peers and teachers see him. The 
writer would interpret the relative superiority of the scale as a 
special instance of a more general principle—namely, that better 
prediction results when the elements in the predictor represent as 
directly as possible the critical elements in the criterion series. 
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THE PREDICTION OF DIFFERENT CRITERIA OF 
LAW SCHOOL PERFORMANCE 


CLIFFORD E. LUNNEBORG anp PATRICIA W. LUNNEBORG 
University of Washington 


Tus study was undertaken to identify (1) those pre-admission 
characteristics of law school applicants which are usefully related to 
law school performance, and (2) those indices of law school perform- 
ance which are most predictable. Current studies of law school suc- 
cess characteristically employ the Law School Admission Test 
(LSAT) and undergraduate grade point average (GPA) as predic- 
tors. Typical results of such studies are those of Lewis, Braskamp, 
and Statler (1964) who reported a correlation of .43 between the 
best weighting of LSAT and GPA with first semester law grades 
for 180 students. Of initial concern in this investigation was the 
determination of other pre-admission factors which might increase 
the predictability of these commonly used indices. Further, as 
Locke (1963) has demonstrated, different criteria may be factori- 
ally independent, with the result that each of several criteria may 
be differentially related to the members of a set of predictors. A 
second concern, then, was whether several criteria of success in 
legal education were associated with the same or different pre-ad- 
missions measures. 


Method 


Subjects 


The complete sample consisted of the 980 students who entered 
the University of Washington School of Law (UW) from autumn 
1956 through autumn 1964. This group was predominantly male 


(96%), single (61%), and had been graduated from Washington 
State high schools (78%). 
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Predictors and Criteria 


All data were taken from the individual files maintained on 
students by the school of law. The 51 predictors listed later were 
derived from the undergraduate grade transcript, LSAT score re- 
port, and law school application form. Undergraduate GPA im- 
provement was defined by subtracting undergraduate GPA from 
senior year GPA. Father’s occupational level and group were coded 
according to Roe (1956). Locations of home, high school, and col- 
lege were coded by state to reflect distance from UW. Sixteen cri- 
teria (also listed later) were taken from the law school transcript 
to span the period of law study and beyond. 


Procedure 


Intercorrelations among the 51 predictors were computed sep- 
arately for several groups of students, including those entering 
between 1956-59, 1956-62, 1956-64. For these same groups, cor- 
relations between the 51 predictors and the criteria were computed. 
These predictor intercorrelations and predictor-criterion correla- 
tions formed the basis for a series of multiple regression analyses 
using the iterative selection technique of Horst (Horst and 
Smith, 1950). 

The results reported in this study tend to concentrate upon the 
1956-59 group. Because this group was subject to less stringent selec- 
tion, it displayed wider variability over the range of pre-admission 
variables than later groups of entrants, which variability is not un- 
like that present in current groups of law school applicants. The 
1956-59 sample thus possessed greater practical predictive signifi- 
cance than later samples. 

In an initial series of analyses a best set of eight predictors was 
selected from among the 51 for each criterion. In the Horst solution 
the best single predictor is first selected, then a second predictor 
which, among those remaining, best complements the first, followed 
by a third which best complements the first pair, etc. As a result of 
these initial analyses, four criteria, ranging from first year law 
GPA through passing the bar exam, were selected for further study, 
and the set of predictors reduced from eight to five, four, and two, 
which were finally assigned weights for establishing predictive 
equations. 
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Results 


Table 1 presents for the four selected criteria correlations with the 
51 predictor variables. Three of these criteria were selected to pro- 
vide immediate, intermediate, and long-term indices which could be 
defined for all law school entrants. First year GPA was selected as 
the most predictable immediate criterion; being dropped for low 
scholarship by the law school was a highly predictable intermedi- 
ate measure of success, and passing the bar exam was the post- 
law school criterion. The fourth criterion, three year cumulative 
law GPA, was included as an example of reliable postdiction, i.e., 
a measure highly related to the predictors but based on a very select 
subgroup of original entrants. The relationship between this crite- 
rion and the pre-admission variables must be interpreted quite 
differently from the first three. 

The single predictors that were most highly correlated with the 
criteria were LSAT total score and the several indices of undergrad- 
uate academic achievement with the exception of grade improve- 
ment. Outside the intellectual realm the variables that were most 
highly correlated with law school success were two reflecting delay 
in entering law school. 

The best sets of eight predictors for the 1956-59 group for all 
16 criteria are presented in Table 2 with an indication of the order 
of selection together with the multiple correlations based on the 
selected sets. These 16 criterion variables were available for all 
students who entered between autumn 1956 and autumn 1959 with 
the following exceptions: second and third year GPA, cumulative 
GPA, and class standing. These six criteria were defined only for 
those who undertook second and third year studies, and conse- 
quently, the interpretation of their relationships to the predictors, 
as cautioned above, must take this into account. The multiple 
correlations (R,) reported in Table 2 and elsewhere in this report 
are corrected coefficients (Snedecor, 1946, p. 348), i.e., they have 
been reduced in size to reflect expected between-sample shrinkage 
owing to sample size and number of predictor variables. 

Over the three law years, the multiple correlations for GPA were 
not appreciably greater than for the purely ordinal class standing. 
Second and third year performances, based on progressively more 
restricted samples, showed little diminution of strength of relation- 
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TABLE 1 


Law School Predictor-Criterion Correlations 
(Decimal Points Omitted) 


Selected criteria 


Three year 
Dropped law 4 
First year by law cumulative Admit 
Predictors law GPA school GPA 


1 Undergraduate GPA 


2 Senior GPA 37 —29 28 
3 Undergraduate GPA improvement 05 —05 —00 
4 Number times on Dean’s List 30 —18 25 
5 Number times on scholastic 

probation —22 26 =14 
6 Number credit hours of accounting 09 —10 09 
7 Accounting GPA? 33 —25 24 
8 Student listed two or more areas 
excelled in 04 —04 02 
9 Student listed one or more areas 
failed -17 19 —17 
10 Entry with B.A. degree —07 14 —03 
11 Attendance at more than one under- 
graduate college —08 10 —08 

12 Major: Pre-Law (A&S) 04 —08 01 

13 Major: Economics 05 ol 07 

14 Major: Political Science 02 —04 —05 

15 Major: Other A&S fields —09 11 —01 

16 Major: Accounting 00 —02 05 

17 Major: Business Administration 01 —04 —06 

18 Major: Engineering 01 —04 06 

19 Major: “Other” professional 

schools (forestry, pharmacy) 01 08 03 

20 LSAT total scores 43 —31 32 

21 LSAT writing ability (N = 341) 26 

22 LSAT general background 

(N = 341) 15 

23 Location of home —04 06 00 

24 Location of high school —06 09 —04 

25 Location of college -01 03 03 

26 Age at entry to law school -17 18 —20 

27 Interval: High school to B.A. or 

entry to law schools -17 13 —22 

28 Interval: B.A. to law entrye -02 10 —01 

29 Sex (male) —02 —02 -11 

80 Father living 04 —03 10 
31 Father an attorney? 03 —03 01 
82 Father some college education 01 01 05 

33 Mother some college education 07 —08 08 

34 At least one attorney relative 01 -01 -07 

35 Financial reliance on others during 

law school 07 —06 —00 

36 Student has one or more dependents —00 07 a 

87 Number of children —02 08 oe 

38 Prior military service —10 10 -14 
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a Indicates some missing data. 


Note.—Variables 8-19, 29-36, 38-41, and 44-51 are dichotomous. 


Selected criteria 
Se LONAS E E 
Three year 
Dropped law 4 
First year by law cumulative Admitted 
Predictors law GPA school GPA to the Bar 
39 Intent to practice law 00 —02 —09 05 
= 40 Intent to teach law —02 05 08 —03 
41 Intent to work part-time during law 
school —08 11 —09 —15 
42 Number hours work/week while in 
law school -11 12 —03 —13 
43 Father’s occupational level (1, pro- 
d fessional through 6, unskilled)+ —06 02 —06 02 
44 Father’s occupation: Social/personal 
service —00 03 00 —08 
45 Father’s occupation: Sales —03 08 06 —05 
46 Father’s occupation: Business, 
organizational, governmental 05 —08 —01 09 
47 Father’s occupation: Technical —03 —02 —02 01 
48 Father’s occupation: Outdoors —01 03 —01 01 
49 Father’s occupation: Science 04 04 03 —03 
50 Father’s occupation: Verbal- 
Cultural 04 —06 —01 04 
$1 Father’s occupation: Performance- 
Cultural —05 02 00 —02 
Law classes included 1956-64 1956-62 1956-62 1956-59 
N 980 750 514 393 
N never less than: 885 677 466 320 


ship; indeed, the three year cumulative GPA based only on those 
subjects completing the three years was as highly related to the 
predictors as first year GPA. Voluntary withdrawal from law school 
was weakly related to the predictors and probably shared little 
variance with other criteria of academic achievement. 

Predictor variables most often included among the first four 
selected, as Table 2 indicates, were LSAT total score, undergraduate 
GPA, interval: high school to B.A. or law school entry, major: arts 

. and sciences fields other than economics, pre-law or political sci- 
ence, entry with B.A. degree, and father’s occupation: business, 
organizational, governmental. 

Eight predictors were selected for further analysis as summarized 
in Table 3. The above-mentioned six were included with age at 
entry substituted for the strongly correlated (.69) high school- 
college completion interval inasmuch as the age measure was the 
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more easily defined and computed. Two other variables, accounting 
GPA and student listed one or more areas failed, were included 
because each was specifically related to two or more of the four 
selected criteria. For each of the four criteria picked for further 
analysis the best sets of five, four, and two predictors were selected 
from among these eight. Standard partial regression weights and 
corrected multiple correlations were determined in each analysis. 

LSAT total score was among the first two predictors selected for 
each criterion. Undergraduate GPA, interestingly, was chosen when 
the criterion was also a continuous GPA; when the criterion was 
dichotomous, a dichotomous measure of undergraduate academic 
success was chosen instead, i.e., student listed one or more areas 
failed. Finally, the set of five selected predictors for each of the 
three predictive criteria was completed by the same three vari- 
ables, age at entry, accounting GPA, and major: other arts and 
sciences fields. 


Discussion 
The most important conclusion from the analyses is that there are 


several items of biographic or educational information which can be 


TABLE 2 


Best Sets of Hight Predictors for Sixteen Law School Criteria 
(N = 393 Entering 1956-59) 


Multiple 
Wr Correlation 

Criteria (R.) Order of predictor selection* 
First year GPA 635 20, 2, 27, 61, 15, 42, 32, 7 
First year class standing 619 20, 2, 42, 15, 7, 9, 51, 29 
Obtained a second year GPA 443 20, 15, 41, 5, 11, 2, 19, 50 
Second year GPA 633 1, 20, 8, 46, 29, 19, 39, 24 
Tio year cumulative GPA 616 1, 20, 46, 19, 24, 8, 13, 7 
Second year class standing 586 1, 20, 46, 27, 19, 7, 8, 24 
ve a third year GPA 463 20, 27, 10, 15, 2, 41, 29, 36 
Saera GPA eas E ao) as, on: 2s) 7 

r > 1, 20, 19, 39, 46, 35, 24, 
Third year class standing 603 1, 20, 19, 24, 39, 28, 7, 34 
m ew member 407 1, 20, 15, 10, 24, 32, 37, 36 

ived “honors” as law oe 

student 444 1, 20, 27, 36, 46, 47, 42, 33 

* ? ? ? 2 ’ 
Admitted to the Bar 468 20, 27, 10, 15, 2, 13, 9, 29 
y from law school 246 41, 29, 16, 35, 45, 25, 4, 11 

ropped by law school 4 521 +26, 9. 45,11, 5, 


20, 2, 26, 9, 45,11, 5, 29 
Graduated from law school 494 20, 27, 10, 5°41 15° 45, 36 


* Predictor identification numbers from Table 1. 
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used together with the LSAT total score and a measure of overall 
undergraduate academic performance to increase the predictability 
of success in law school. For each of the four criteria selected for 
final study, multiple correlations based upon five variables were 
significantly greater than those utilizing only the two measures. 
Furthermore, the additional contributing variables can be the same 
whether the predictive criterion is immediate, intermediate, or long- 
term. 

Two of the five selected predictors merit additional comment. 
In a related study (Lunneborg and Lunneborg, 1966), first year 
law grades were found to be negatively related to those delays in 
education which occurred prior to attainment of the baccalaureate 
degree, but were unrelated to those occurring after the baccalaure- 
ate. Because age at entry indexes both of these kinds of delay, its 
predictive value may be attenuated. Its practical value in admis- 
sions formulae over calculating periods of delay is hoped to offset 
this loss. The predictor variable, major: other arts and sciences 
fields, serves to dichotomize law school entrants into those whose 
undergraduate program was one of professional preparation (engi- 
neering, business administration, forestry, etc.) or preparation 
for a legal profession (pre-law, economies, political science) and 
into those who majored in some less professionally-oriented arts 
and sciences subject such as English, history, or physics. It is tempt- 
ing to look upon this variable as one which distinguishes those 
with professional educational goals from those who sought a liberal 
arts education. 

Variables of interest that did not prove to be of predictive value 
included the LSAT part scores (general background and writing 
ability) and improvement in undergraduate grades (senior year GPA 
minus cumulative GPA). Although LSAT part scores could not be 
included in the 1956-59 analyses reported here, they were part of 
the pool available for predicting first year law grades for entrants 
between 1960-64. Neither part score was ever included among the 
first eight predictors selected. The undergraduate grade improve- 
ment measure was developed because of the belief in law school 
admissions committees that later undergraduate work is more valid 
than earlier work. It was thus hypothesized that students who 
showed a negative discrepancy between their senior year and cu- 
mulative GPA would perform most unsatisfactorily in law school, 
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and that those who showed little or no discrepancy would be less 
apt in law study than the “late bloomers” who should show a large 
positive difference when their cumulative GPA was subtracted from 
their senior year GPA. This hypothesis was not supported. 

The strong relationship between pre-admission variables and three 
year cumulative GPA is noteworthy for several reasons. Rather 
than providing a basis for predicting success in law school for all 
entrants, it represents an explanation (a postdiction) of the three 
year success for the survivors in the system. Because of the attrition 
which occurred prior to completion of the third year, the magni- 
tude of the relationship which exists for this postdictive criterion 
is quite surprising. Differences among the more select third year 
students were apparently just as closely related to pre-law vari- 
ables as the differences among the first year students. Distefano 
and Bass (1959) observed similarly among a highly restricted 
sample of practicing lawyers that LSAT and pre-law grades discrim- 
inated between those rated as high and low in legal ability. 


Summary 


A series of multiple regression analyses was undertaken to iden- 
tify pre-admission variables useful in predicting several criteria of 
law school success. LSAT total scores and undergraduate academic 
performance were consistently the strongest predictors. Age at en- 
try, accounting grades, and undergraduate major were shown to in- 
crease the predictability not only of first year law school GPA but 
also of continuance in school and eventual passage of the bar ex- 
amination. 
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VERIFICATION OF SIX PERSONALITY FACTORS 


ANDREW L. COMREY anp KAY JAMISON 
University of California, Los Angeles 


Previous factor analytic personality research papers in this se- 
ries have been based upon the assumption that it is desirable to 
factor analyze total scores over collections of relatively homogene- 
ous items, rather than to analyze single items (Comrey, 1961, 
1962, 1964, 1965, 1966; Comrey and Schlesinger, 1962). Items have 
been developed to fit a certain homogeneous item pool, but have 
been required to exhibit a statistical as well as a logical belonging 
before being actually assigned to that item group. 

On the basis of these previous findings, six major personality 
factors were selected for further study. Six factored homogeneous 
item dimensions (FHIDs) were chosen to represent each hypothe- 
sized factor. Each FHID, in turn, consisted of six items intended to 
be relatively homogeneous in content. Most of the FHIDs selected 
for a given factor had been good marker variables for that factor 
in previous analyses; some were newly developed for this study. 
The major purpose of the present investigation was to determine 
whether the postulated six-factor personality structure would 
emerge as predicted. There were also two secondary objectives. The 
first of these was to determine whether the expected factor structure 
could be obtained despite an important departure from previously 
used sampling procedures. The second minor objective was to ob- 
tain some information about the effect of equating the number of 
positively and negatively worded items in each FHID. In previous 
investigations, no systematic attempt was made to control this pos- 
sible source of bias. 
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Procedure 
Sample 


The personality inventory prepared for this study was distrib- 
uted to an area sample of people in Van Nuys and Santa Monica, 
California. Both of these areas are middle-class communities with 
individual sub-areas ranging in economic status from lower to up- 
per middle class. The sample was drawn by numbering all the 
blocks in the two areas and then choosing ninety numbers at ran- 
dom from a total of 1570 possibilities. Four of these blocks were 
rejected because they were non-residential. Every home on a se- 
lected block was visited to explain the study and to invite partici- 
pation. To encourage participation, an individual personality anal- 
ysis was promised to each participant. The sample used consisted 
of the first 274 answer sheets returned, 153 females, and 121 males. 
The mean age was 39.4 and the mean number of years of schooling 
completed was 13.2. The respective standard deviations were 16.3 
and 2.5. In previous samples, respondents were not contacted per- 
sonally but merely given a written invitation to participate. Fur- 
thermore, a substantial part of each sample consisted of university 
students. 

In the present sample, a total of 1152 dwelling units was visited; 
500 units visited had no one at home; 234 units refused to partici- 
pate; 418 units produced at least one person willing to participate. 
A total of 737 inventories was distributed in this manner but only 
274 were completed and returned. The most common reasons given 
for refusal to participate were: not interested, not enough time, too 
old, too sick, and language problems. Suspicions about the nature of 
the project, misgivings about who would see the results, antipathy 


toward psychology, and antipathy toward UCLA were less fre- 
quently mentioned reasons. 


Inventory 


The inventory consisted of 216 items designed to measure six 
hypothesized factors: Shyness, Dependence, Empathy, Neuroticism, 
Compulsion, and Hostility. Thirty six items were allocated to each 
factor, divided among six FHIDs of six items each. For example, 
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the Dependence factor comprised the following six FHIDs: Suc- 
corance, Lack of Self Sufficiency, Deference, Conformity, Need for 
Approval, and Affiliation. Under each FHID were six items which 
were expected to correlate substantially with each other. In addi- 
tion, each FHID designed to measure Dependence was expected to 
correlate with each other such FHID. 

In an attempt to achieve higher item reliability than that typi- 
cally obtained with forced-choice and true-false item formats, two 
response scales were used which had nine alternatives. Scale X had 
the following possible answers: 1. Never, 2. Almost Never, 
3. Rarely, 4. Occasionally, 5. Fairly Often, 6. Frequently, 7. Usu- 
ally, 8. Almost Always, and 9. Always. Scale Y had these possible 
responses: 1. Absolutely Not, 2. Very Definitely Not, 3. Definitely 
Not, 4. Probably Not, 5. Possibly, 6. Probably, 7. Definitely, 8. 
Very Definitely, and 9. Absolutely. The subjects were informed that 
they could use either scale but next to each item number on the 
inventory an X or a Y was marked to suggest the scale most 
probably suitable. 

Half the 216 items were worded positively with respect to the 
FHID name and half were worded negatively. For example, two 
items on the FHID, Reserve, were, “I do less than my share of the 
talking in a conversation,” and “I do more than my share of the 
talking in a conversation.” These two items were well-separated in 
the inventory booklet, as were all items measuring the same 
FHID. An earlier investigation found a measure of acquiescence to 
have little relationship to these factors (Comrey, 1964). For pur- 
poses of further investigation, however, the FHIDs in this study 
were composed to control this possible source of bias. 

The 36 FHIDs used in this investigation are listed below with 
a sample item from each. A reliability estimate is given in paren- 
theses. This estimate was obtained in each case by averaging the 
interitem correlations to obtain an estimate of a one-item test and 
then correcting this figure by the Spearman-Brown formula to a test 
of the number of items used. Items follwed by an asterisk are neg- 
atively worded with respect to the dimension name. 

1. Seclusiveness (.85) IX. I dislike being with a lot of people I 
don’t know. 

2. Succorance (.79) 20X. I dislike being dependent on someone 
else.* 
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3. Generosity (.70) 75X. I am willing to share what I can with 
others less fortunate. 
4. Inferiority Feelings (.77) 76X. I feel inferior to the people I 
know. 
5. Grooming (.84) 23X. I am unconcerned about how my clothes 
look to others.* 
6. Hostility (.78) 24Y. Most people are valuable human beings.” 
7. Reserve (.86) 79X. I have little to say in a group. 
8. Lack of self-sufficiency (.86) 8Y. I dislike being left alone. 
9. Service (.73) 81Y. I would like a job in which I help people 
who have problems. 
10. Depression (.82) 82X. I feel that life is drudgery and boredom. 
11. Cautiousness (.86) 29X. I like to live dangerously.* 
12. Rhathymia (.64) 84Y. I think it is better to enjoy life than to 
do something worthwhile for society. 
13. Shyness (.89) 13X. I find it difficult to talk with a person I 
have just met. 
14. Deference (.63) 86Y. I would like a regular job as an assistant 
to a really great man. 
15. Helpfulness (.79) 15Y. I enjoy helping people even if I don’t 
know them very well. 
16. Pessimism (,86) 88Y. I am inclined to be a pessimist. 
17. Order (89) 17X. I keep everything in its proper place so I 
know just where to find it. 
. Cynicism (.81) 180Y. Most students in school would rather 
fail than cheat.* 
19. Stage Fright (.80) 109Y. It would be hard for me to do any- 
thing in front of an audience. 
20. Conformity (.67) 38X. I feel better doing what everyone else 
is doing. 
» Interest in People (.72) 183Y. I would prefer a job where 
dealing with people is the most important part of it. 
22. Agitation (86) 130X. I relax without difficulty.* 
. Love of Routine (.76) 41X. I like to maintain a regular sched- 
ule of daily activities, 
. Psychopathy (80) 114Y. If I were in business, I would lie if 
necessary to make money. 


Submission (.76) 205X. I manage to make my presence felt 
in a group.* 


N 
= 


[S] 
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26. Need for Approval (.83) 62X. I ignore what my neighbors 
might think of me.* 

27. Sympathy (.77) 45X. I am a very sympathetic person. 

28. Inadequacy (.78) 208Y. I think I have a lot of ability.* 

29. Meticulousness (.78) 119X. When I do a job, I try to make it 
perfect down to the last detail. 

30. Defensiveness (.71) 66X. When I have to compete with other 
people, I find them to be friendly and fair.* 

31. Follower Role (.81) 193X. I refuse to compete in struggles for 
power. 

32. Affiliation (.63) 50X. I am loyal to my friends even when they 
are wrong. 

33. Tolerence (.65) 123X. I like to associate with people who come 
from a very different background than my own. 

34. Lack of Ego Strength (.83) 142X. I feel able to deal with the 
problems I face.* 

35. Drive to Finish (.86) 215X. It is difficult for me to keep at 
something until it is finished.* 

36. Aggression (.74) 144X. I try to avoid saying anything that 
might make somebody mad.* 


Analysis 


Preliminary item analyses were carried out to determine which 
items should be retained for the final analysis of FHIDs. The items 
were divided into six groups of 36, with all the items from one 
FHID in the same group. Each factor had a FHID in every group 
of 36 items. Each group of 36 items had one and only one set of six 
items from every hypothesized factor. A factor analysis of items 
was carried out for each of these six groups of 36 items. In all of 
these analyses, a factor emerged as expected for every FHID in- 
cluded. Some of the factors were less clear than others, however, 
and several items had to be dropped because they failed to achieve 
sufficiently high loadings on the appropriate item factor. Fifteen of 
the FHIDs had all six items retained, 10 had five items retained, 
seven had four items, three had three items, and one FHID had 
only two of the original six items retained. Total FHID scores 
were computed by adding up the scores on the retained items. 
Negatively worded items were reversed by subtracting the item score 
from 10 before adding. The intercorrelation matrix of Pearson co- 


950 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


efficients was computed for these 36 FHID scores. To check for the 
presence of alternate forms among the variables, high values in the 
correlation matrix were corrected for attenuation to determine if 
they would approach 1.0; in no case did this occur. 

The 36 X 36 matrix of uncorrected correlations among FHID 
scores was factor analyzed by the minimum residual method (Com- 
rey and Ahumada, 1964, 1965). Twelve factors were extracted be- 
fore encountering convergence on vectors of opposite sign, estab- 
lishing 12 as the upper limit on the number of valid factors. These 
12 factors were rotated orthogonally by the normal criterion I of 
the Tandem Criteria for analytic rotation in factor analysis (Com- 
rey, 1967). Only six major factors appeared in the criterion I ro- 
tations. These six criterion I factors were further rotated by the 
normal criterion II of the Tandem Criteria. Criterion I attempts to 
crowd the variance on as few factors as possible, subject to the 
limitation that variables on the same factor must be correlated. 
This makes it easier to identify and eliminate minor factors. After 
eliminating the minor factors, the remaining factors are rotated 
by criterion II which approximates simple structure where the data 
permit such a solution. Criterion II accomplishes this by rotating 
to reduce the loadings of uncorrelated variables on the same 
factors.t 


Results 


Loadings of 3 or more on the six criterion II factors will be 
given below. If a variable was hypothesized to appear on the fac- 
tor, its loading will be given even if it is less than .3. If a variable 
was not hypothesized to be on the factor, the factor loading will be 
followed by an asterisk. 

I. Shyness. 1. Seclusiveness, .63; 7. Reserve, .59; 11. Cautious- 
ness, .35;* 13. Shyness, .62; 19. Stage Fright, .65; 25. Submission, 
48; 31. Follower Role, .70; and 36. Aggression, —.41.* 


1The computations for this study were carried out on the IBM 
J 7094 op- 
ET A UCLA Computing Facility. The following materials have beat 
es oe the ADI Auxiliary Publications Project, Photoduplications 
ES rary of Congress, Washington, D. C. 20540: Correlation matrix, 
a ae factor matrix, criterion I rotations, criterion II rotations, 
te nh answer sheet, and list of items for each FHID and factor. A 
Sr A secured by citing document number 9053 and remitting in ad- 
Me or 35 mm. microfilm or $2.50 for photoprints. Make check payable 
: Chief, Photoduplication Service, Library of Congress. 
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II. Dependence. 2. Succorance, .53; 8. Lack of Self Sufficiency, 
.52; 14. Deference, .28; 20. Conformity, .67; 26. Need for Approval, 
.53; and 82. Affiliation, .40. 

III. Empathy. 1. Seclusiveness, —.38;* 3. Generosity, .70; 9. 
Service, .55; 15. Helpfulness, .74; 21. Interest in People, .65; 27. 
Sympathy, .70; and 33. Tolerance, .38. 

IV. Neuroticism. 2. Suecorance, .30;* 4. Inferiority Feelings, 
.64; 10. Depression, .73; 16. Pessimism, .71; 22. Agitation, .66; 
25. Submission, .37;* 28. Inadequacy, .79; and 34. Lack of Ego 
Strength, .77. 

V. Compulsion. 5. Grooming, .40; 11. Cautiousness, .48; 12. Rha- 
thymia, —.36;* 17. Order, .69; 23. Love of Routine, .53; 29. Me- 
ticulousness, .61; and 35. Drive to Finish, .61. 

VI. Hostility. 6. Hostility, .64; 12. Rhathymia, .50; 18. Cyni- 
cism, .73; 24. Psychopathy, .59; 30. Defensiveness, 51; and 36. 
Aggression, .36. 

Six criterion I factors were regarded as too minor to include in 
the criterion II rotations. Three of these had no loading as high as 
.8. Loadings of .3 or more will be given below for the remaining 
three residual factors. 

Residual. 11. Cautiousness, .37; and 23. Love of Routine, .32. 

Residual. 5. Grooming, .41; and 17. Order, .33. 

Residual. 30. Defensiveness, .37. 


Discussion 

In examining the results, it is clear that six major personality 
factors did emerge as hypothesized. The remaining six factors were 
very minor in importance by comparison with the first six. Every 
FHID expected to appear on a given factor, except Deference on 
the Dependence factor, did have a loading of at least .3 on that 
factor. Neuroticism was loaded by all six of its defining FHIDs to 
the extent of .5 or more; Shyness, Empathy, and Hostility had five 
out of six FHIDs with loadings of .5 or more; and Compulsion 
and Dependence both had four of their six FHIDs with loadings of 
.5 or more. No FHID had a loading in excess of .41 on a factor it 
was not intended to define. The few loadings of .3 or more for 
FHIDs on factors they were not intended to define were reasonable. 
The negative loading of —.41 for Aggression on the Shyness factor 
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and the loading of —.36 for Rhathymia on the Compulsion factor 
are both in line with what might be expected. 

A statistical comparison of the mean factor scores for males and 
females in this sample is given in Table 1. These factor scores 
were obtained by computing the total scores over those items re- 
tained for each FHID on the basis of the item analyses. Thus, scores 
for a given factor were obtained by adding the total scores for those 
FHIDs designed for that factor. In most cases, the retained items 
were those which had loadings of at least .45 on the item factor 
which emerged for that FHID in an analysis of items. The FHIDs 
Rhathymia and Affiliation, however, did not produce good item fac- 
tors. For the Rhathymia FHID, four items were selected which had 
the highest intercorrelations. These intercorrelations ranged from 
.27 to .35. For the Affiliation FHID, two items were selected for 
retention which correlated 48 with each other. The matrix of cor- 
relations among the six major factor-score variables is given in 
Table 2. 

With reference to the primary objectives of this investigation, 


TABLE 1 
Means and Standard Deviations for the Factor Scores 
Males Females 

Factor M SD M SD t 
Shyness 82.9 19.7 85.1 22.0 86 
Dependence 99.4 19.4 106.5 21.5 2.82** 
Empathy 95.9 16.7 104.4 14.8 4,44** 
Neuroticism 61.5 15.9 66.3 20.6 2.11* 
Compulsion 141.4 19.9 141.5 21.1 40 
Hostility 83.5 19.1 80.0 20.2 1,45 

* Significant at the .05 level. 
** Significant at the .01 level. 
TABLE 2 
Correlations among Factor Scores 
1 2 3 4 5 

1 

2 lL 

3 —.24 28 

4 35 -30 —.08 

5 16 04 .08 —.23 

6 -1 = .22 — AA 27 —.32 
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it seems evident that the postulated six-factor personality structure 
did emerge as predicted. Since these factors have been identified in 
several previous studies in this series, it seems reasonable to de- 
scribe them as well-defined and consistent factored measures of per- 
sonality. Furthermore, the factor structure emerged despite a sub- 
stantial departure from previous sampling procedures, suggesting 
that these factors have some degree of stability over different, 
kinds of subject populations. Finally, the conversion of items to an 
even split between positively and negatively worded statements 
resulted in essentially the same factors as those obtained when the 
statements were all worded in one direction. 
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THE DEVELOPMENT OF A SCALE TO MEASURE 
ATTITUDINAL DIMENSIONS OF THE EDUCATIONAL 
ENVIRONMENT! 


JOHN K. TUEL 
Oral Roberts University 
AND 
MERVILLE C. SHAW 
University of California, Los Angeles 


A large scale research project was initiated in 1963 for the pur- 
pose of testing a rationale for guidance services which postulated 
that the prime objective of the guidance specialist should be to 
effect positive changes in the educational environment (Shaw and 
Tuel, 1964a). Two sub-objectives of the study were: (1) to assess 
the nature and attitudinal structure of the educational environ- 
ment (Tuel, 1964a,b), and (2) to attempt to modify the attitudes 
of significant persons in the educational environment (Shaw and 
Tuel, 1964b). Essential to both of these objectives was the utili- 
zation of a broad-spectrum attitude inventory capable of adminis- 
tration to various educational role groups including students, par- 
ents, teachers, guidance specialists, and administrators. The major 
purpose of such an inventory would be to reflect basic attitudes to- 
wards educational philosophies and practice. 

Review of the relevant literature revealed that the topic had 
received but scant attention from research workers, and that such 
an inventory meeting the above specifications did not then exist. 
Necessity then led to the development, described below, of the 
School Opinion Survey (Tuel and Shaw, 1964), a scale designed to 


1 This research was a facet of a larger project supported by a grant from 
the National Institute of Mental Health administered through the Interpro- 
fessional Research Commission on Pupil Personnel Services, 
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meet both the specific needs of the current research project and 
what seemed to be a more general need for an instrument to assess 
the education-related attitudes of the significant persons in the 
learning environment. 

The School Opinion Survey was built upon several assumptions. 
The first was that the “educational environment” is a useful con- 
struct describing that part of the total effective environment which 
directly influences academic learning (Tuel, 1964b). The second was 
that the educational environment exhibits several levels extending 
from the intrapersonal (Tuel and Wursten, 1965b) through a wid- 
ening scope to the national level (Tuel and Wursten, 1965a). A 
third assumption was that philosophy, values, objectives, and tech- 
niques form a continuum or hierarchy from the abstract to the con- 
crete, from the philosophical to the operational, or, looking at it 
still a third way, from the attitudinal to the behaviorial. A fourth 
postulate was that there is some degree of congruence among the 
philosophy and values espoused by an individual, the objectives 
which he sees as being appropriate for education, and the tech- 
niques he favors in carrying out educational procedures. Finally, 
it was proposed that attitudes along this continuum and their de- 
gree of congruence could be measured. 


Procedure 


i A pool of over three hundred items reflecting various philosoph- 
ical positions, values, educational objectives, and techniques for 
accomplishing these objectives was assembled and tried out on 
various small samples. Items which proved ambiguous or which 
showed a restricted range of responses were dropped from this pool. 
Since the questionnaire was designed for use with the entire gamut 
of educational role groups, i.e., with parents, teachers, administra- 
tors, and counselors at all school levels, as well as with students 
at the high school level, vocabulary level and meaningfulness had to 
be carefully controlled. Preliminary screening left a pool of 250 
items. These items were then administered to all teachers, princi- 
pals and counselors, and to all tenth grade students and their par- 
ents in a medium sized California city school district. The result- 
ing data were subjected to factor analysis with varimax rotation. 
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Results and Discussion 


Ten prominent factors were extracted, which together accounted 
for over 70 per cent of the total variance. The first three of these 
factors appeared to represent the major positions in educational 
philosophy, as outlined by Brackenbury (1959) and subsequently 
shown by Tuel (1966) to have been operative in the development 
of American higher education. The remaining seven factors repre- 
sented techniques for implementing various patterns of educational 
philosophies. 

The first and strongest factor contained items chiefly concerned 
with individual differences, individual personal development and 
human objectives as opposed to strictly intellectual subject-mat- 
ter-oriented objectives. Educational objectives stressing self- 
realization of the consequences of one’s own behavior, personal 
enjoyment of learning, development of critical thinking, and 
breadth of curriculum tended to fall into this factor. All these ob- 
jectives presupposed a philosophical structure which saw reality as 
centered in the human being as an individual and a value system 
focused on the enhancement of the individual personality. This fac- 
tor has thus been named the Humanist factor. 

Factor II appeared qualitatively to be the antithesis of Fac- 
tor I, i.e., a sort of “antihumanist” scale. However the two factors 
actually proved to be uncorrelated (See Table 1). Factor II stressed 
intellectual development, attention to “objective facts,” avoidance 
of controversial issues, and the irrelevance to education of individ- 
ual differences in feelings, interests and motivation. Such values 
evidently stemmed from a philosophical system which located re- 
ality outside of the individual in ideas, society, or the material 
universe, i.e., idealism, social relativism, or realism. Since the ex- 
ternal “objective” component seemed strongest, it was decided to 
designate this the Realist factor. 

Factor III presented an interesting pattern. The scale was com- 
posed of two equal parts of opposite valence. Half of the items took 
a social relativist or experimentalist position stressing the relativ- 
ity of knowledge and truth, the irrelevance of religion to education, 
and the social origin and changeability of moral law. The other 
half of the items (which were all negatively correlated with those 
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of the first half) described a theistic or absolutist position, empha- 
sizing the immanence of God as the source of truth and moral law. 
There would have been equal justification for naming Factor III 
the “Theist” or “Absolutist” scale, on the one hand, or the “Rel- 
ativist” or “Experimentalist” scale on the other. Because the last 
seemed the most appropriate option in an educational setting, the 
scale was named the Hxperimentalist factor. 

As mentioned above, the first three factors of the School Opinion 
Survey appeared to describe three major dimensions of educational 
philosophy: the humanist, realist, and experimentalist. The inter- 
correlation between these three factors was nil; i.e., they seemed to 
represent orthogonal dimensions. In contrast to the first three fac- 
tors, the remaining seven exhibited a fair degree of intercorrelation, 
while preserving their distinction as separate factors. These factors 
all represented the fourth plane of the theoretical model, the plane 
of educational techniques, i.e., the specific methods advocated to 
realize educational objectives. It is of interest to note that none of 
the technique items loaded significantly into any philosophical 
scale during factor analysis. This suggested that the educational 
techniques which people advocate may have much less relation- 
ship to the philosophy, values, and objectives they espouse than 
most would like to believe. 

The seven technique scales were named respectively, Individual 
Attention, Group Activities, Professionalization, Non-Academic, 
Academic Discipline, Scientific Objectivity, and Strict Control. The 
first of the technique scales, Factor IV, stressed (1) individual at- 
tention and counseling and (2) closer home-school and parent- 


TABLE 1 
School Opinion Survey—Scale Intercorrelation Matrix 


2 3 4 5 6 7 8 9 10 Factor Scale Name 


.04 oo 30 —.18 —.02 —.22 13 — 14 I Sip 

: He ap -14 —.06 11 —.18 III Experimentalist 

i :52 .49 49 89 .47 IV Individual Attention 
08.33 —.24 26 —.22 V Group Activities 

-33 .40 43 36 VI Professionalization 
.02 .38 .03 VII Non-Academic 

.20 .52 VIII Academic Discipline 

-31 IX Scientific Objectivity 


TUEL AND SHAW 959 


teacher relationships. Advocated were individual counseling on 
personal problems, individual attention to pupils and encourage- 
ment of creativity, parent orientation, parent-teacher conferences, 
home visits by teachers, school social workers, and better training 
of counselors. Common to all these characteristics was the con- 
cept of increased attention given by school staff to individual 
students in their school, classroom, and home educational environ- 
ments, i.e., the “pupil personnel orientation.” This scale correlated 
with the Humanist scale (.33) and negatively with the Realist 
scale (—.24). It also correlated well with each of the technique 
scales except Factor V. (Group Activities). It was decided to des- 
ignate Factor IV the Individual Attention factor. 

The fifth factor contained items emphasizing competitive and 
team sports, co-educational physical education, outdoor play and 
field trips, extracurricular activities and group projects, grading on 
the curve, PTA activities, and local school board autonomy. The 
outdoor, athletic, non-academic and small group emphasis were 
apparent. Any individual focus was conspicuously absent. Of all 
the technique factors, this correlated best with the Realist scale 
(.30). It also correlated with Non-Academic (.33) and Scientific 
Objectivity (.26) and negatively with Academic Discipline (—.24) 
and Strict Control (—.22), to be described below. Factor V was 
named the Group Activities factor. 

Factor VI items advocated larger school districts and the aspects 
of staff professionalization usually associated with them: higher 
salaries for teachers and administrators, clerical help for teachers, 
educational research, school psychologists, and individual attention 
programs to assist the emotionally disturbed pupil and to en- 
courage the gifted. It is not surprising to learn that Factor VI cor- 
related with the Humanist scale and even slightly with the Ex- 
perimentalist scale. Among the technique scales, it correlated well 
with Individual Attention, Academic Discipline and Scientific Ob- 
jectivity, yet to be described. It correlated least with Group Ac- 
tivities and negatively with the Realist scale. Factor VI was des- 
ignated the Professionalization scale. 

Factor VII was composed of items stressing nonacademic and 
practical curriculum elements such as student government, shop 
and craft classes, art and music classes, consumer education, and 
other techniques aimed at preventing dropouts and retaining the 
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interest of poorly motivated or non-academically oriented students, 
e.g., better lighting and library, methods courses for teachers, and — 
haying incidental expenses of education assumed by the school. 
The essence of this factor seemed to be the special provisions in 
curriculum, school plant, services, and class presentation necessary 
to interest and retain non-academically oriented students. This 
factor correlated with both the Humanist and Experimentalist 
scales and also with Individual Attention, Group Activities, Pro- 
fessionalization, and Scientific Objectivity. It was uncorrelated 
with the Realist scale, Academic Discipline and Strict Control. 
Factor VII was named the Nonacademic scale. 

Factor VIII items advocated increased time spent in study: more 
hours in the school day, weeks in school year, longer class periods 
and less free time. Also stressed were “solid” highly verbal college- 
preparatory type subjects involving concentrated study: foreign 
language, social studies, writing of themes and emphasis on great 
literature. This factor correlated positively with the Humanist scale 
and negatively with the Realist scale. Among the technique factors 
it correlated best with Strict Control, then with Individual At- 
tention, Professionalization, and (negatively) with Group Activi- 
ties. It was uncorrelated with Nonacademic. It seemed most ap- 
propriate to term this the Academic Discipline factor. 

Factor IX was composed of items concerned with relatively im- 
personal scientific teaching methods: use of teaching machines, 
team teaching, stress on mathematics and use of personality, 1Q, 
standardized and objective tests. Also included were state regula- 
tion of education, free medical care for students, and child-study 
training. This factor exhibited low positive correlations with all 
three philosophy scales and among the technique scales correlated 
in descending order with Professionalization, Individual Atten- 
tion, Nonacademic, Strict Control, Group Activities, and Academic 
Discipline. The core concept of this factor was somewhat more dif- 
ficult to determine than was that of the others, However, since the pre- 
dominant element was the use of impersonal scientific educational 
methods, it has been designated the Scientific Objectivity scale. 

The last factor was concerned with the theme of strict discipline 
and moral training: strict enforcement of school rules, strictness of 
aaa: spanking of misbehaving pupils, stringent laws against 

Y, teaching of morals, and self discipline in school. Corporal 
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punishment was advocated and the belief expressed that punish- 
ment usually produces the desired results. This factor correlated 
moderately with the Humanist and negatively with the Realist 
and Experimentalist factors. Tt also correlated in descending order 
with Academic Discipline, Individual Attention, Professionaliza- 
tion, and Scientific Objectivity. There was no relationship with 
Nonacademic and moderate negative correlation with Group Ac- 
tivities. It seemed most appropriate to name this the Strict Con- 
trol factor. 

Tt is evident that Individual Attention is the most pervasive of 
the technique factors. It stems from a Humanist (and anti- 
Realist) philosophical position. This approach to education seems 
to be expressed in various ways: through increased Professionaliza- 
tion of the staff, special provision for Nonacademically oriented 
students, Strict Control of the fractious, and even to some degree 
through the application of Scientific Objectivity. 

Over against Individual Attention (and practically uncorrelated 
with it) is Group Activities. Here the emphasis is on the social con- 
text of individuals rather than on any one individual himself; 
hence the correlation with the Realist and, even slightly, with the- 
Experimentalist philosophical positions. The impersonal contextual 
emphasis of Group Activities is reflected in its correlated factors: 
Nonacademic and Scientific Objectivity. As expected, it correlates 
negatively with Academic Discipline and Strict Control. One is 
tempted to term this factor the “Playboy Orientation,” as one of its 
major themes is to make school a sort of game. 

Thus, there seem to be two major technique “families,” distin- 
guished primarily by whether the focus is on the individual stu- 
dent and his personal development or on the group context, group 
interaction and play. These orientations seem to be related, al- 
though imperfectly, to the humanist and realist philosophical po- 
sitions respectively. A second but less prominent division signified 
by zero intercorrelation appears to lie between the Nonacademic 
factor on the one hand, and the factors of Academic Discipline 
and Strict Control on the other. The positive correlation of the 
former and negative correlations of the latter pair with the Group 
Activities and Experimentalist factors hint of the participation 
of the age-old relativist-absolutist philosophical controversy in this 
dimension. 
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Implications 

Besides meeting the experimental necessity which gave it birth, 
the School Opinion Survey offers a means of measuring and describ- 
ing diagnostically attitudinal differences among educational role 
groups. It is hypothesized that such data can help unearth the 
causes of such problems as relatively poor general academic per- 
formance, low morale, or poor human relations within a school 
system and/or poor rapport between a school system and its com- 
munity. When a marked difference is found between the values or 
preferred techniques to which a school system, on the one hand, 
and the majority of parents in its community, on the other, sub- 
scribe, certain signs of stress may be anticipated. When a new 
principal takes over for another whose values and preferred tech- 
niques were far more congruent than are his with those of his 
faculty and the parents of that attendance area, difficulty may be 
just off-stage. It is conceivable that the availability of such meas- 
ures will make possible not only the ferreting out of some causes of 
already developed difficulties but also the prediction and preven- 
tion of unnecessary values-techniques conflicts before they become 
entrenched. Data are at present being collected which will, it is 
hoped, shed light on the validity of these hypotheses. These data 


are expected to become available for publication during the coming 
year. 
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NEED-PRESS AND EXPECTATION-PRESS INDICES 
AS PREDICTORS OF COLLEGE ACHIEVEMENT! 


CARL G. LAUTERBACH an» DAVID P. VIELHABER 
Medical Research Project, West Point, N. Y. 


Wuen a college applicant chooses a particular educational in- 
stitution it may be assumed he does so largely because he expects it 
will most adequately satisfy his personal needs. If he is accepted he 
may or may not find the directive influences or “press” of the col- 
lege in keeping with his needs or his expectations of the institution. 
It has been hypothesized that the extent of agreement (congruence) 
between one’s internal forces (needs) and the external environ- 
mental forces (press) he encounters is positively related to his 
adaptation to that environment (Stern, 1960; Pace, 1961). This 
may be referred to as the need-press hypothesis. It may be, how- 
ever, that it is not necessarily so much the congruence of his needs 
and the press as it is the congruence of what he expects (expecta- 
tions) and the press he subsequently encounters that more strongly 
influences his adaptation. This may be referred to as an expecta- 
tion-press hypothesis. 

These hypotheses involve abstract concepts, i.e., press, needs, 
and expectations, and the defining and measurement of their ref- 
erents pose certain difficulties; Getzels and Guba (1957), for ex- 
ample, have effectively stressed the need for establishing distinc- 
tions, classifying such terms, and stating their interrelationships 
with behavior. Of the three terms, the press of collegiate environ- 
ments has perhaps been most successfully defined and measured, by 
such instruments as the College Characteristics Index (CCI) (Pace 
~i Impetus for this study stemmed directly from the original ideas and un- 
finished work of the late John P. Devlin; the authors are deeply indebted for 


his pioneering efforts. The study was supported by the U. S. Army Medical 
R & D Command and the United States Military Academy. 
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and Stern, 1958). However, research suggests that press alone is 
insufficient to predict students’ achievements (Stricker, 1965), so 
that it may be more promising to utilize concepts such as need- 
press and expectation-press hypotheses to develop indices which 
include comparable representation of both internal forces (needs 
and expectations) and the press of the environment. 

The purpose of this study was to make a comparative examina- 
tion of the need-press and expectation-press hypotheses by em- 
pirically investigating need-press and expectation-press indices as 
predictors of adaptation of West Point cadets. These indices were 
derived by comparing profiles of the College Characteristics Index 
obtained from cadets under differing conditions. 

Although it may be difficult to agree upon a sole criterion of a 
student’s success in college, certainly in any collegiate environment 
academic achievement is a principal aspect of success. In addition, 
recognizing that a student’s nonacademic performance is also an 
integral part of his total college development and achievement, 
West Point has developed a system whereby cadets are regularly 
rated by their peers and superiors regarding their leadership po- 
tential. In this study these Aptitude for Service Ratings, which 
have been found to be reliable measures and the best single pre- 
dictor of later success as an officer (Crockett and Bowen, 1961), 


were considered in addition to grades as criteria of college achieve- 
ment. 


Method 


Test Measures 


The CCI, administered under special conditions, was used to ob- 
tain measures of new cadets’ needs and expectations. It was also 
used to gauge the press of the institution which it traditionally 
measures. The same instrument was used for all of these purposes 
in order to diminish the problem of finding parallel scales from 


different instruments for comparisons of needs, expectations, and 
press, 


Subjects and Conditions 


383 cadets, the N or need group, 


were administered the CCI 
entrance to the Academy under ee ai 


special instructions to describe 
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West Point only as they preferred it to be (preference condition). 
Since they presumably had no first-hand experience with the in- 
stitution, their individual CCI profiles were considered expressions 
of their desires or needs specifically related to their choice of col- 
legiate environment. 387 other cadets, the E or expectations group, 
were also administered the CCI, but under the usual standard in- 
structions to describe their institution as they saw it. Since this 
second group also had virtually no first-hand experience with West 
Point, their individual CCI profiles were considered expressions of 
their expectations. A measure of the press of the institution was de- 
rived from the mean CCI profile of 646 cadets from an earlier Plebe 
(freshman) class. They had completed the CCI under standard in- 
structions midway through their Plebe year after they had ex- 
perience with the environment. They are refererd to as the P or 
press group. 


+ 


Need-Press Indices 


For each of the 383 Ss in the N group, two indices of need-press 
congruence were calculated. First, for a measure of the similarity of 
the shape of his need profile with the press of the environment, his 
30 CCI scale scores were correlated with the mean scale scores of 
the P or press group. This correlation coefficient is referred to as 
his r N-P score. As an estimate of the distance between an S’s 
needs and the press of the environment, & D statistic (Cronbach 
and Gleser, 1953) was calculated between his 30 scale scores and 
the mean scale scores of the P group. This is referred to as his 
D N-P score. 


Expectation-Press Indices 


Similarly, for the 387 Ss in the E group, two indices of expecta- 
tion-press congruence were calculated. His r E-P score, an esti- 
mate of the similarity in shape of his expectations profile to the 
press of the environment, was the correlation coefficient between his 
30 CCI scale scores and the 30 scale means of the P group. His 
D E-P score, an estimate of the distance between his expectations 
profile and the press, was the D statistic between his 30 scale scores 
and the scale means of the P group. 
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The following indicators of USMA adaptation were obtained for 
all subjects in the N and E groups: 


Academic Performance Criteria 


Grade Point Average. An S’s average for all “academic” sub- 
jects, excluding physical education and strictly military subjects. 
This was obtained for his Plebe (freshman) year (GPA-1) and for 
his Second Class (junior) year (GPA-2). 

Tactics Average. Grades for military subjects. They were desig- 
nated TA-1 for Plebe year and TA-2 for Second Class year. 


Nonacademic Performance Criteria 


Aptitude for Service Ratings. Estimates of an S’s leadership po- 
tential based on ratings by his peers and superiors. They were ob- 
tained for the Plebe Year (ASR-1) and for the Second Class year 
(ASR-2). 

Physical Education Average. This measure was also obtained for 
8’s Plebe year (PE-1) and the Second Class year (PE-2). 


Results and Discussion 


Neither of the need-press (r N-P and D N-P) nor the expecta- 
tion-press (r H-P and D N-P) indices were correlated significantly 
with the nonacademic criteria (Physical Education average and 
Aptitude for Service Ratings) for either the Plebe or Second Class 
years. Therefore, as tested here, neither the need-press nor expec- 
tation-press hypotheses were useful in the prediction of non-class- 
room forms of cadet achievement. 

However, positive support for the expectation-press hypothesis 
Was gained from the correlations of the r E-P and D E-P indices 
with the four academic criteria (GPA-1, GPA-2, TA-1 and TA-2) ; 
all eight of these correlations were positive, and six were statisti- 
cally significant. These results are shown in the first two columns 
of Table 1. None of them, however, was high; the coefficients 
ranged from 24 between r E-P and TA-1 to .11 between r E-P and 
GPA-2. The D E-P index appeared to have the most consistent 
validity of the two expectation-press indices, in that it correlated 
reliably with each of the four academic criteria, whereas the r E-P 
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index related significantly only with the two Tactics averages, 
TA-1 and TA-2. These results suggest that closeness in the distance 
between expectations and press profiles has more significance for 
predicting the academic criteria than does the similarity of their 
shapes alone. To summarize, the closer the expectations profile 
was to the press profile at West Point, the better an S’s subsequent 
academic achievement tended to be. Stated otherwise, the greater 
his accuracy or insight into the Plebe year press, the slightly 
greater the likelihood of his excelling in academic performance. 

Of the corresponding eight correlations of the need-press indices 
(r N-P and D N-P) with the academic criteria, five were sig- 
nificant (p < .05). These results are reported in the last two col- 
umns of Table 1. However, the need-press indices correlated with 
each of the academic criteria in the opposite direction from that 
predicted from the need-press hypothesis! That is, all eight corre- 
lations were negative in sign, ranging front —.21 between the r N-P 
index and GPA-2, to —.06 between T N-P and TA-1. Thus, the 
less congruent an S’s CCI profile of preferences or needs was with 
the press, the better his academic achievement tended to be. Stated 
more directly, new cadets who preferred the West Point educational 
climate most differently from how experienced cadets reported it 
tended to be successful academically. Incongruity in the shape of 
the need and press profiles appeared to be about as predictive of 
grades as the distance between them, since both of the need-press 
indices correlated at the same significance level (p < .01) with 
GPA-1 and GPA-2. 


TABLE 1 


Correlations of Expectation-Press and Need-Press Indices 
with Freshman and Junior Year Academic Grades 


Index 
he le Pea ar 
r E-P D E-P rN-P DN-P 
Criteria (N = 272) (N = 263) 
GPA 
Freshman 12 :19%* —.17** —.15* 
Junior wll 15* —.21** —.20** 
Tactics Grades 
Freshman .24** .228* —.09 — .06 
Junior BHT? .15* —.13* —.09 
*p < .05. 


“p < 01. 
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Since the need-press and expectation-press indices were surpris- 
ingly different in their relationships with academic criteria, an ef- 
fort was made to understand better their meaning by determining 
their correlations with eight well-explored West Point selector 
variables: high school rank; an index of high school extracurricular 
activities; and SAT-Verbal, SAT-Math, Mathematic achievement, 
English composition, and physical aptitude test scores. The final 
selector variable used was the CEER score (College Entrance Ex- 
amination (high school) Rank, which is a multiple regression com- 
posite of the five selector variables that best predicts West Point 
academic achievement. These results are shown in Table 2. 

The correlations of the expectation-press indices with the eight 
selector variables were all positive, but only their correlations with 
SAT-Verbal (r’s of .23 and .26) and English composition (r’s of 
-21 and .23) were significant (p < .01). In contrast, both need- 
press indices tended to be negatively associated with SAT-Verbal, 
SAT-Math, and CEER scores, and were virtually unrelated to the 
other five selector variables. Only three significant correlations 
among the sixteen between need-press indices and selector vari- 
ables were found: —.14 between r N-P and SAT-Math; —.15 be- 
tween D N-P and SAT-Verbal; and —.15 also between D N-P and 
SAT-Math scores. 

Since each of the expectation-press and need-press indices cor- 
related in the same direction, and to about the same extent, with 
certain of the selector variables as they had with the academic 


TABLE 2 


Correlations of Expectation-Press and Need-Press 
Indices with Standard Selector Variables 


Index 
; rE-P D E-P - - 
Selector Variable (N = 272) i Nw = 208) $ 
High School Rank (Av.) 
Extra-Curricular : oa 02 03 05 
pepe 126## “ogee A e 
E -03 10 =a —.15* 
Mathematics or a z rs a ‘07 
Eng. Comp. (214+ ogee <09 ‘00 
Physical Apt. Test 01 05 03 <03 
* p <05. : 


p< 01. 
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criteria, it seemed likely that they were measuring certain cogni- 
tive factors in common with the selector variables. Such interpre- 
tation of the nature of the indices was supported by obtaining 
multiple correlations predicting academic grades using the CEER 
composite selector score in combination with the expectation-press 
and need-press indices. The results indicated that the correlations 
between CEER and Grade Point Average, which ranged from .42 
to .63, could not be raised appreciably by combining any of the 
expectation-press or need-press indices with CEER. However, 
CEER alone correlated only .30 with TA-1 and .25 with TA-2, 
but when the r B-P index was combined with CEER, the resulting 
multiple correlation reached .38 for TA-1 and .30 for TA-2. Thus, 
it appears that the r E-P index may tend to contribute some unique 
variance to the specific prediction of Tactics (military subjects) 
grades not accounted for by the presently used selector variables. 
However, in general it appears that the expectation-press and need- 
press measures do not aid appreciably in the prediction of grades 
from presently available selector variables. 

It may be concluded that the ability of a new cadet to anticipate 
accurately the press at West Point appears to be a function of cog- 
nitive factors. His initial knowledge of the educational environ- 
ment at West Point can thus be viewed as a part of general verbal 
knowledge and ability which are also reflected in his SAT-Verbal 
and English Composition test scores. 

However, the tendency of new cadets to express preferences 
which are incongruous with the actual press of the environment 
reported by experienced cadets was also related to cognitive factors 
as measured by selector variables. How cognitive factors operate 
in cadets’ expression of their preferences relative to the West Point 
environment is not clear. It may be that intellectually outstanding 
cadets who score highest on particular selector variables (SAT- 
Verbal and SAT-Math) simply happen to have need-states which 
are most different from the press, but this need-press disparity is 
not relevant to their performance in the manner specified by the 
need-press hypothesis. Indeed, the empirical findings of this study 
indicate that close congruence between needs and press is associated 
with lesser academic achievement. 

Perhaps a value of this study lies in its helping to denote some 
of the behavioral correlates of student expectations and needs or 
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preferences, required in order to comprehend the role of such con- 
cepts in affecting behavior within an institution, as urged by Get- 
zels and Guba (1957). The results may also be suggestive of possible 
value in increasing efforts to provide early realistic orientation to 
new students, at least to West Point cadets, concerning the institu- 
tional realities which they will subsequently experience. 


Summary 


Half of an entering Cadet Class described West Point on the 
College Characteristics Index (CCI) as they saw it (Expectations), 
and half were instructed to describe it as they preferred it 
(Needs). The individual CCI profiles in each group were compared 
with the mean CCI profile of 646 experienced cadets (Press), re- 
sulting in individual need-press and expectation-press congruence 
indices. Correlations of these indices with academic grades lend 
support to the expectation-press hypothesis, but for the need-press 
hypothesis the results were opposite from those predicted. Rela- 
tionships of the congruence indices with usual college selector vari- 
ables suggest that they may be principally cognitive measures. 
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A COMPARATIVE STUDY OF PERCEPTIONS 
OF A UNIVERSITY ENVIRONMENT BETWEEN 
HONOR AND NONHONOR FRESHMEN GROUPS 


SHELDON R. BAKER 
Wisconsin State University, River Falls 


Present studies of college and university environmental char- 
acteristics indicate that major plans of study, “level of training 
sought (Thistlewaite, 1960), and changes in level of aspirations 
(Thistlewaite, 1966) are significantly related to environmental 
press. Level of grade expectation and decisiveness in vocational de- 
cision making are also found to be significantly related to the per- 
ception of press (Baker, 1965). 

In the above studies as in those of Astin and Holland (1961), 
Astin (1962), Stern (1963a,b), and Pace (1963), college environ- 
mental characteristics are conceptualized as global variables ca- 
pable of shaping the productivity of academic institutions and as 
capable of affecting the decisions of students in their competition 
for limited academie rewards. This study accepts the paradigm 
which defines environmental press as & global factor, but only as 
it bears upon indigenous productivity, such as the election of an 
honors program, and the maintenance of academic status. 

The election of an honors program by students generally indi- 
cates a strong need to learn as well as a need for intellectual rec- 
ognition. Do students who elect. an honors program perceive the 
press of their university significantly different as compared to those 
who do not select such a program? This study will attempt to con- 
sider the foregoing question by comparing the perceptions of en- 
vironmental press between honors and nonhonors students. 

With this purpose in view, the following null hypothesis is for- 
mulated for research: ; 
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There are no differences in the perception of environmen 
press between freshman honors program participants and fresh- | 
man nonhonors participants. ; 


Procedures 


Fifty-seven second quarter freshmen honors program partici- 
pants were administered the Stern’s College Characteristics Index. — 
Within one week another group of thirty-three freshmen non= 
honors students volunteered to take the Index. The second group 
were part of a randomly selected number of one hundred students 
who were invited by letter to participate in the study. The second — 
group of students had not elected an honors program nor were they 
asked by the administration to elect the program. i 

Thirteen factor scores were calculated for each student. The 
mean scores for each group were’ converted to standard scores with 
a mean of 0 and a standard deviation of 2. Appropriate t tests of 
significance were used to evaluate the null hypothesis, and the .05 — 
level of significance was accepted as the minimum for the rejection i 
of the null hypothesis. 


TABLE 1 


Standard Score Difference in Press Perception between Honors 
and Nonhonors Students, N = 90, M = 0, SD = 2 


pu J EE ane ern O 


Nonhonors Honors 
Factors (N = 33) (N =57) Diference ~ 
Intellectual Climate Factors 
Work-Play 5 —2.0 2.5% 
Non-Vocational Climate -1.3 u —.2 
Aspirational Level -2.1 -1.0 1.0* 
Intellectual Climate -1.8 —2.0 2 
Student Dignity 2.5 -1.6 .o* 
x catenio Climate -2.0 —2.0 0.0 
on-Intellectual Climate Factors 
Academic Achievement -2.3 -1.7 -6 
Self Expression -2.6 -1.2 -m 
troup Life —.2 8 —1.0* 
demic Organization —:3 -7 4 
Play-Work vi 2:0 -io 
Vocational Climate 14 11 3 
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Results 


The major findings are summarized in Table 1. As can be seen 
from the data, the null hypothesis is rejected for differences be- 
tween the following factors: 


Work-Play oo reae Marais oe cee een ete ete .001 level 
Aspiration Level: .:.isscensis'sulsabinaiee insti la 01 level 
Self Expression’... emsoss aesan EENEN AIN oe .001 level 
Group Life .01 level 
Play-Work .01 level 
Student Dignity > sss cesses AAA E ee 01 level 


Honor students tend to perceive greater strength on Aspiration 
Level, Student Dignity, Self Expression, Group Life, and Play- 
Work as compared to nonhonor students. 

The data also indicate that the learning environments within 
the same institution differ for honors as compared to nonhonors 
students. 


Implications 


The findings do not support the model of press as a global factor 
but suggest that an environment may contain multiple sets of press 
which become manifest upon the demands of salient needs of stu- 
dents in achieved status positions. Perhaps the variables which are 
purported to be measured by the College Characteristics Index are 
subject to the same types of influence as are social judgments and 
other types of social perceptions. The study recommends further 
elaboration with regard to the construct validity of the College 
Characteristics Index. 
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THE VALIDITY OF A COMPREHENSIVE COLLEGE 
SOPHOMORE TEST BATTERY FOR USE IN 
SELECTION, PLACEMENT, AND ADVISEMENT 


THOMAS M. GOOLSBY, JR. 
The Florida State University 


Ir has become increasingly important in the past decade to pro- 
vide for comprehensive testing at the college sophomore level on 
several counts. First, it is going to be almost imperative that se- 
lection occur for students entering the upper division. Secondly, 
it is very important to advise and guide students into appropriate 
curriculums of study in the upper division. Thirdly, there is the 
matter of shaping the curriculum of the junior college and the lower 
division of other institutions of higher learning. 

At the present time there is no single test, battery of tests, or 
total testing program to provide adequate information for intel- 
ligent decision-making to meet any of the needs alluded to above. 

It has been the experience of the admissions personnel and aca- 
demic advisers that there is a dearth of adequate information for 
their needs. In most cases an estimate of success in school is based 
on grades or grade point averages, which are subsequently used as 
criteria. From grades in high school or in the lower division of the 
University, predictions of grades in future academic work have 
been made. From the scatter-plots of these relationships, expect- 
ancy tables have been developed for use in selection, placement, 
and advisement. Such expectancy tables have been based on single 
variable relationships rarely exceeding .60 and multiple-correlation 
Coefficients rarely exceeding .75. It has not been infrequent that re- 
searchers have called for a thorough examination of the validity 
of grades or the grade point average. Intuitively, most researchers 
Tecognize that grades have poor measurement characteristics and 
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that any statistical manipulations based on them lead to many 
kinds of erroneous conclusions. According to Yonge (1965): 

Many investigators have been moving away from the old 
model of predicting intellective criteria (e.g., grades) from in- 
tellective predictors (e.g. ability tests) .. . 

Although grades are still a popular criterion, many investigators 
have displayed an interest in non-intellective predictors or cor- 
relates of grades... 
Goodstein and Heilbrun found that personality factors con- 
tribute most to the prediction of the academic achievement of 
the average college male... 
Very little or no research effort has been made to develop a sub- 
stitute measure for grades or a measure nearly so good as any of 
the predictors frequently used. 

On the national scene, reputable measurement specialists have 
been concerned with sophomore testing and have been baffled by 
the complexity of the problems involved in making provisions for 
comprehensive measures. Among the experimental instruments is 
the Comprehensive College Tests (COT) published by Educational 
Testing Service. 

The Comprehensive College Tests were developed to provide a 
flexible program of college achievement testing to be used for & 
variety of purposes. Among these purposes are: evaluation of in- 
dependent study for college credit, college equivalency, transition 
to upper-division studies, curriculum evaluation, and institutional 
self-study. The program of Comprehensive College Tests consists of 
the General Examinations and the Subject Examinations. 

The General Examinations provide a comprehensive measure of 
undergraduate achievement in the five basic areas of liberal arts 
education: English Composition, Humanities, Mathematics, Na- 
tural Sciences, and Social Sciences-History. 

An experiment was designed to attempt to determine to what 
extent the General Examinations of CCT could be used for (a) 
a eee 
the upper division at a lar, ee oe aig omar 

ge southeastern university. 


Procedures 


pals was administered to currently enrolled second-semester 
phomores in March, 1964. Florida Twelfth Grade Achievement 
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Tests (FTGAT) scores, School and College Ability Tests (SCAT) 
g scores, and Sophomore Cumulative Grade Point Averages (Soph 
4 CGPA) were also collected in March, 1964. FTGAT and SCAT 
were administered in October, 1961. The Soph CGPA scores were 
collected by subject-matter area and total. Junior Year Grade 
Point Averages (Junior GPA) were collected a year later. 
Intercorrelations among all measures were obtained. These in- 
tercorrelations were obtained for two purposes: to determine the 
measurement characteristics of CCT and to determine the utility of 
CCT as a predictor of certain criteria, including Grade Point Av- 
erage (GPA). 


Results 


Table 1 indicates the correlations between CCT subtests. These 
relationships closely approximate those reported in other studies 
involving the use of CCT. The correlations between the subtests of 
Natural Science (.68), Mathematics (.64), and Humanities (.65) 
are of a magnitude indicative of some degree of independence. 
Without very much question, the subtests in these three areas are 
justifiable. The correlation between the subtests of the Social Sci- 
ences and History Test (.84) is considerably high. One may ques- 
tion the inclusion of these separate subtests in the CCT Battery. 
Subtests with such a degree of relationship may be justified on the 
basis of a desire to emphasize a variety of important objectives in 
the curriculum and to facilitate follow-up activities after the tests 
are taken. For purposes of selection and placement, however, the 
use of both of these tests is questionable. 


TABLE 1 
Correlations between CCT Subtests (N = 700) 
CCT Subject Tests Correlation 

1. Natural Sciences 

Physical Sciences 

Biological Sciences -68 
2. Mathematics 

Basic Skills 

Content and Concepts 64 
3. Humanities 

Fine Arts 

Literature 65 
4. Social Sciences and History 

Social Sciences 

Histo: 84 


— Ory Ooie 
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From the intercorrelations presented in Table 2, one can im- 
mediately determine that there is a fair degree of independence 
among all tests of the CCT. Although these interrelationships were 
quite low, they were all significantly different from zero. As ex- 
pected, the greatest degree of independence was between Mathe- 
matics and Humanities (.15), and the greatest amount of overlap 
was in English and Humanities (.56). The range of coefficients, 
however, was not very great (.15 to .56). In general, the degree of 
independence indicated here is somewhat unusual for achievement 
tests and highly desirable when the test battery is considered as a 
collection of entities. 

In general, Table 3 indicates something of the predictive power 
of CCT. The highest relationship between Soph CGPA and any 
given subtest of CCT was .45. Even though this degree of relation- 
ship is significant, it has relatively little practical value for pre- 
dictive purposes. When Junior Year GPA was used as a criterion, 
the highest relationship with any given test of CCT was .25. For 
practical uses, the tests of the CCT do not serve as a good pre- 
dictor of Junior GPA or Sophomore CGPA. 


TABLE 2 


CCT Intercorrelations (N = 700) 
OOOO O O 


ccT Natural Social 
Subject Tests English Sciences Mathematics Humanities Sciences 
See eee eS anheg Soenen. 
English .38 32 31 
Natural Sciences "53 a “49 
Mathematics 115 ‘81 


Humanities 44 
eee ee 0 0 Na 


TABLE 3 
Correlations of Certain Predictors of GPA (N = 700) 


k Soph 
Predictor Variables OGPA ory 

SCAT Total 

12th Grade Test Ga a 
CCT English ‘45 ‘25 
CCT Natural Sciences 30 “15 
COT Mathematics "30 ETI 
CCT Humanities 43 “23 
CCT Social Sciences 33 3B 
Soph CGPA vs Junior GPA 3s 


á 
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Included in Table 3 is the relationship between Soph CGPA and 
Junior Year GPA (.58). The reduced magnitude of the relationship 
was somewhat due to the small variance of scores for the Sopho- 
more CGPA. 

As indicated in Table 4, the relationship between CCT and 
FTGAT was .75. One would consider such a relationship to be of 
some practical value. The relationship is interesting, since, for all 
intents and purposes, both test batteries were designed to measure 
instructional outcomes. This degree of relationship may be ac- 
counted for by considering the rigorous structuring of the tests ac- 
cording to specifications and the standard conditions under which 
they were administered. With a general improvement in testing 
and grading practices for courses taught in the University, one 
may hopefully expect a relationship between any of the instruments 
in Table 4 to correlate with GPA at least as high as the correlations 
among the tests; i.e., at least .70. When SCAT was used as a pre- 
dictor of FTGAT, Soph CGPA, or CCT, the relationships were 
respectively .73, .50, and .70. SCAT does not predict Soph CGPA for 
any practical use. It does, however, predict FTGAT and CCT with 
a fair degree of efficiency. Here again, one may surmise that rigor 
in constructing and administering tests is a probable contributor 
to the higher relationships. 

When certain tests of CCT were combined in a multiple predic- 
tion procedure as indicated in Table 5, the predictive efficiency im- 
Proves very little. 

Tables 6 and 7 present an analysis of CCT when subject area is 
of major concern. The means, standard deviations, and reliabilities 
of the CCT tests for the experimental group and the national 
norming group are presented in Table 6. In general, the means for 
the experimental group were slightly higher than the means for the 


TABLE 4 
Correlations of Certain Tests and of CCT 
with Soph CGPA (N = 407) 
. - ee) Ss S 


SCAT Total vs FTGAT 73 

SCAT vs CCT -70 

CCT vs FTGAT 75 
SRR ee i a E, 


CCT vs Soph CGPA -50 
~ BRE Ee Sy a 
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TABLE 5 
CCT Multiple Correlations with GPA (N = 700) 


Soph. GPA 
Combinations of Predictor Variables CGPA Jr. Year 
CCT English + CCT Humanities 45 40 
CCT Mathematics + CCT Humanities .53 .24 
CCT Natural Sciences + CCT Mathematics .35 .21 
CCT English + CCT Math + COT Social Sciences .45 
COT Nat. Sci. + CCT Math. + CCT Social Sciences 83 


national norm group. The standard deviations were in general, 
much lower for the experimental group than for the national norm 
group. Reliability coefficients reported in the manual range from 
91 to .95 for the tests of the CCT. 

As indicated in Table 7, the coefficients of correlation between 
CCT and certain subject areas are quite low for any practical pre- 
dictive use. 

A superficial inspection of a scatter plot, not presented here, will 
indicate something of the inefficiency of CCT to predict Soph 
CGPA. Approximately an equal number of students who achieved 
below a GPA of 2.01 were equally distributed at each score from 
324-606 on the CCT English. When considering students who 
made a GPA of 2.01, there is still one chance in three that the true 


TABLE 6 
Means, Standard Deviations, and Reliabilities for CCT 
CCT Subject Tests 
oe as NS rial al hs 
: Eng, NS. Ms. Hs. SS & Hy 
ee eee ese Hs | BO BY 
Mean 
Exp. 512.4 503.3 475.7 
f h 530.0 515.1 
£ a 498 498 498 499 498 
Exp. 76.7 76.0 
? y 54.7 74.8 75.5 
7 Nat. 99.0 99.0 99.0 99.0 99.0 
Nat — Narona aia P 8 Prosent Study 
Eng. — English 
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TABLE 7 


Correlations between Parts of the CCT and GPA 
by Subject Area (N = 407) 


GPA Criteria 
CCT Subject Test Eh. NS. Ms. Hs. SS & Hy. Soph 
Predictors GPA GPA GPA GPA GPA CGPA 

CCT Eh, 56 .28 .19 -20 .34 .39 
CCT N.S. .18 .36 .05 .12 .12 .26 
CCT Ms. .08 .33 -27 .04 .23 27 
CCT Hs. AT 22 05 16 .53 .28 
CCT SS & Hy. .10 .22 .05 .16 .53 .28 

GPA — Grade Point Average 

Eh. — English 


N.S. — Natural Sciences 
Ms. — Mathematics 
Hs, — Humanities 

SS —Social Sciences 
Hy. — History 


Soph CGPA is greater than 2.42 or less than 1.60. This magnitude 
of error in prediction leaves much to be desired in making any sort 
of practical decision about selection, placement, and advisement. 


Summary and Conclusions 


A comprehensive sophomore college testing program is becoming 
imperative for selection, placement, and advisement purposes. 

The CCT has satisfactory measurement characteristics on most 
counts and excellent ones on a few. The evidence presented in this 
paper does not support the use of the CCT for selection, placement, 
and advisement at the sophomore level when it is considered alone 
and especially when GPA is a criterion. Furthermore, there is a 
need to question the desirability of the use of GPA as a criterion. 

There is evidence in this paper to support a need for a very sub- 
stantial emphasis on adequate measurement and grading practices 
within the University. 

A rigorous determination and definition of curriculum and the 
construction of criterion measures (cognitive and affective) in a 
continuing research program is necessary for higher education to 
meet its responsibilities in selection, placement, and advisement. 
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MULTIPLE DISCRIMINANT PREDICTION OF 
COLLEGE CAREER CHOICE 
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Rusearcu studies have revealed that there exist distinct personal- 
ity patterns which characterize specific professional groups (Izard, 
1960; Roe, 1946a, 1946b, 1951a, 1951b, 1953; Schaffer, 1953; Siegel- 
man and Peck, 1960) as well as specific academic groups preparing 
for various professional fields (Merwin and DiVesta, 1959; Garrison 
and Scott, 1961; Stern, 1962; Stern, Stein and Bloom, 1956; Zuck- 
erman, 1958). Prior investigations have emphasized the differenti- 
ation and description of these professional and academic groups. 
Few studies though, have attempted to predict group membership, 
either professional or academic, based on the personality dimensions 
they have described. 

The present study departed from the more frequently adopted 
approach of univariate analysis and, utilizing mulitple discrimi- 
nant analysis, explored the feasibility of predicting students’ aca- 
demic choice based on personality need constructs as measured by 
Stern’s Activity Index. 


Method 


Sample 


Three male groups representing three academic areas of study, 
business, chemistry, and mathematics, and two female groups, edu- 
1The authors wish to express their appreciation to the members of the 


Applied Mathematics Division, Esso Research and Engineering Company, 
and in particular to J. Beckwith, M. Efroymson, R. Hardy, and R. Heitler. 
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cation and nursing, served as criterion groups. These groups were 
comprised of a total of 245 students, 50 in each group, with the ex- 
ception of chemistry which contained 45 students. Two additional 
male groups, 32 business and 36 mathematics students, and two ad- 
ditional female groups, 32 education and 38 nursing students, were 
utilized as cross-validation samples to test the classification ef- 
ficiency of the discriminant functions derived from the criterion 
groups. Although the groups were pre-selected according to their 
area of study, students were assigned randomly to the criterion 
and cross-validation samples. The students shared several char- 
acteristics in common. They were all Caucasian, native-born 
American, approximately 21 years of age and, with the exception 
of the nursing students, all were matriculated, day students at- 
tending a large metropolitan university. Nursing students were 
drawn from a metropolitan hospital-affiliated nursing school. At 
the time of testing, all students were seniors, and all had indicated 
that they intended to pursue the professional career for which they 
were then preparing. Students who had changed their major aca- 
demic area of study more than once, who had changed majors be- 
cause of academic difficulty, who had indicated dissatisfaction with 
their choice of major or who were not maintaining satisfactory 
academic standing were eliminated from the sample, 


Predictor Variables 


Subjects were administered the Activities Index (AI) in group 
sessions lasting approximately one hour. The AI consists of 300 
items describing commonplace daily activities and feelings which 
are keyed to 30 of the Murray needs, There are 10‘items for each of 
the 30 measured needs to which the subject is required to respond 
for each item with a “like” or “dislike” choice (Stern, 1958). All 
80 variables were used in the analysis, 

Three multiple discriminant analyses? were computed, one for the 
three male groups of business, chemistry, and mathematics students, 
one for the same business and mathematics groups (omitting chem- 
istry students), and one for the two female groups of education and 


2 Th j = 
by G a ere ee Was carried out by means of DISCRIM, a program developed 


Jones (1904) Poles (1962) and adapted for use with the IBM 7094 by 
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nursing students. Classification was based on the derived discrimi- 
nant functions.® 


Results 


The means, univariate F tests, and scaled vectors are summa- 
rized in Table 1 for the male business, chemistry, and mathematics 
groups. The descriminating power of the predictor test battery was 
determined by computation of Wilks’ lambda, which is a function 
of the roots of WA, where W is the pooled within-groups devia- 
tion scores cross products matrix, and A is the among-groups 
matrix of cross products of deviations of group from grand means 
weighted by group sizes (Cooley and Lohnes, 1962; Rao, 1952). 
The effectiveness of this discrimination was significant (F = 2.25, 
df = 60/225, p < .001). Chi square tests (Bartlett, 1941) were 
computed for each of the two derived discriminant functions (by 
removal of successive latent roots) to determine the significance of 
discrimination along each dimension (Jones, 1964). The first dis- 
criminant function was found to be significant (x? = 120, p < 
001), but the significance of the second vector failed to reach the 
necessary level (p = .11). The first vector accounted for 72 per 
cent of the predictable group variation. 

The relative contribution of the predictor variables to each of 
the discriminant functions may be seen from an examination of the 
scaled weights (Table 1). These weights were derived by multipli- 
cation of the normalized latent vectors by the square root of the 
corresponding diagonal element of the W matrix. The scaled 
weights for the first discriminant vector indicate that the predic- 
tors providing the greatest contribution were determined by the 
business group's high scores for need affiliation, exhibition, and hu- 
manism; the chemistry students’ high need scores for practicality 
and science; and the mathematics students’ high need scores for 


succorance. 
Since the second vector proved to be nonsignificant, difficulty 


was encountered not only in interpretation of factors, but also in 


later classifications. Another discriminant analysis was then com- 
puted for the same criterion group of business and mathematics 
students, omitting the chemistry group. For this analysis, the ef- 


3 Classification was accomplished by means of CLASSIF (Cooley and Lohnes, 


1962), adapted for use with the IBM 7094 by Jones (1964). 
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fectiveness of the discriminating power of the test, Wilks’ lambda, 
was significant (F = 2.56, df = 29/70, p < .001) as was the dis- 
criminating power of the one discriminant function* computed 
(x? = 60.78, p < .001). The function accounted for 100 per cent 
of the predictable group variation. 

A review of the scaled weights summarized in Table 2 indicates 
that many of the predictor variables which contributed to the 


TABLE 1 


Means, Univariate F Tests, and Scaled Vectors for Male Business, 
Chemistry and Mathematics Groups 


Means Scaled Vectors 
Needs Bus Chem Math F* p I II 
e S S S gs Rt A 
Abasement 8.72. 3.82 3.98 .25 ns .08 .09 
Achievement 6.388 5.67 5.38 2.50 .08 05 —.06 
Adaptability 4.92 5.22 4,92 .33 ns —.09 —.15 
Affiliation 7.10 5.69 6.12 . 4,46 .01 .18 —.03 
Aggression 5.22 4.64 4.06 3.06 .04 .00 —.18 
Change 5.26 4.27 4.70 2.95 .05 —.06 .07 
Conjunctivity 4.60 6.09 5.50 5.06 .007 -—.06 —.O1 
Counteraction 6.50 6.383 6.20 27 ns .10 (nae 
Deference 5.36 6.11 6.08 2.19 ns —.l4 —.04 
Dominance 6.64 6.36 5.386 4.01 .O1 —.09 —.25 
Ego Achievement 5.80 4.96 3.96 5.36 .005 02.04 
Emotionality 3.18 3.58 3.88 1.95 ns —.21 —.06 
Energy - j 8.20 6.33 6.84 1.53 ns —.02 27 
itionism 2 3.78 : ` à 5 +02 
Henne 3.28 4.51 .01 18 0; 
Achievement 5.18 4.78 3.88 3.99 .02 01 —.38 
Harm Avoidance 3.26 4.82 4.52 5.62 004 —.09 +03 
Humanism 5.12 5.09 4.20 1.92 ns 40 —.24 
Impulsiveness 5.74 4.13 4.90 8.79 .0003 16 -20 
Narcissism 4.68 4.42 4.78 29 ns -06 -09 
Nurturance 5.34 5.38 5.46 -03 ns 05 —.08 
Objectivity 8.60 8.24 8.56 1.11 ns 17 -10 
o 4.08 5.78 4.74 5.03 .007 06 —.36 
ie 6.04 4.96 5.54 2.45 .08 —.05 —.29 
Practicality 5.68 6.24 5.72 94 ns -15 —.01 
Petesiyeuees 6.06 6.89 6.46 1.72 ns —.02 10 
Eee 8.58 7.81 5.90 25.17 0001 —.71 | —.19 
nsuality 4.26 4.18 4.54 .55 ns .02 -24 
a 3.96 4.20 4.84 1.71 ng —.13 -10 
anepi ; 5.60 6.20 680 4.69 0l —.23 22 
inderstanding 6.54 6.93 7.30 1.53 ns 00 25 
* df = 2/142, 


Bus = Business 
Chem = Chemistry 
Math = Mathematics 


ee Ea 
7 be kth 4 
The number of discriminant functions computed is g-1, where g = groups. 
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TABLE 2 


Means, Univariate F Tests, and Scaled Vectors for Male 
Business and Mathematics Groups 


Means Scaled Vectors 
Needs Bus Math Ft p I 

Abasement 3.72 3.98 48 ns —.01 
Achievement 6.38 5.38 5.62 -01 .14 
Affiliation 7.10 6.12 5.23 .01 -21 
Aggression 5.22 4.06 5.56 .01 .13 
Change 5.26 4.70 2.00 ns —.07 
Conjunctivity 4.60 5.50 3.47 .01 —.03 
Counteraction 6.50 6.20 51 ns .07 
Deference 5.36 6.08 2.74 .01 —.13 
Dominance 6.64 5.36 7.76 .006 .09 
Ego Achievement 5.80 3.96 10.64 .001 —.02 
Emotionality 3.18 3.88 3.86 04 —.10 
Energy 6.90 6.84 .03 ns —.09 
Exhibitionism 4.78 3.28 9.03 .003 —.01 
Fantasized 

Achievement, 5.18 3.88 7.39 007 +26 
Harm Avoidance 3.26 4.52 6.75 01 —.09 
Humanism 5.12 4,20 3.13 .07 -50 
Impulsiveness 5.74 4.90 4.60 .03 wll 
Narcissism 4.68 4.78 04 ns —.15 
Nurturance 5.34 5.46 06 ns 17 
Objectivity 8.60 8.56 04 ns .19 
Order 4.08 4.74 1.44 ns 29 
Play 6.04 5.54 1,19 ns .02 
Practicality 5.68 5.72 01 ns .07 
Reflectiveness 6.06 6.46 -79 ns —.21 
Science 3.58 5.90 17.34 -0001 —.35 
Sensuality 4.26 4.54 63 ns —.10 
Sex 3.96 4.84 3.23 .07 —.01 
Succorance 5.60 6,80 9.19 -003 —.25 
Understanding 6.54 7.30 3.11 .07 —.31 


Note.—Because of programing difficulties, only 29 need variables were utilized in this analysia. 
* df = 1/98. 
Bus = Business 
Math = Mathematics 


differentiation between the three male groups were also responsible 
for the differentiation between the business and mathematics groups 
for this analysis. The mean need scores for the business and math- 
ematics students remain the same in this analysis. When consider- 
ation was given to the means of all three male groups (Table 1), 
the mean scores of the chemistry students tended to fall between 
those of the business and mathematics students. When the mean 
scores for the chemistry students are no longer considered (Table 
2), the contrast between the business and mathematics students 


990 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


becomes more apparent. The scales furnishing the greatest relative 
contribution were determined by the business group’s high scores 
for needs affiliation, aggression, dominance, exhibition, fantasized 
achievement, humanism, and play, and the mathematics group’s 
high scores for needs order, science, succorance, and understanding. 
The ambivalence of the motivations of the business students was 
suggested by the findings that these students appear to seek out 
close, friendly relationships, while at the same time expressing 
feelings of aggression and ascendency over others. Their interest in 
people would seem to be a reflection of their own need for self- 
gratification, since their social participation expresses their needs 
for self-display and attention seeking. Achievement motivation 
tends to be on a fantasized level revolving around daydreams of 
success in achieving extraordinary public recognition. In contrast, 
mathematics students receive satisfaction in imposing order and in 
manipulating their environment symbolically. These students 
would seem to have a greater need for detached intellectualization 
through utilizing problem solving or theorizing as an end in them- 
selves. In interpersonal situations, though, the mathematics stu- 
dents’ affectional needs are satisfied by dependency on others. 

Table 3 summarizes the results obtained with the two female 
groups. Wilks’ lambda proved to be significant (F = 2.07, df = 
30/69, p < .006) as did the discriminating power of the discrim- 
inant function (x = 53, p < .005). The function accounted for 
100 per cent of the predictable group variation. The scaled weights 
indicate that the following variables yielded the greatest relative 
contributions: dominance, ego achievement, energy, harm avoid- 
ance, humanism, narcissism, objectivity, and sex. Female education 
students displayed greater needs for dominance or ascendency, & 
striving for power, fearfulness, interest in the humanities and so- 
cial sciences, narcissistic trends, and objectivity which was in con- 
trast to the nursing students who displayed greater needs for ac- 
tivity and heterosexual interests. The fact that two need variables, 
narcissism and sex, with insignificant univariate F tests, contrib- 
uted to the discriminant function, may be explained by the value of 
employing multivariate techniques. The contribution of these vari- 
ables was relatively high in terms of the factor loadings for the 


total profile (see Cooley and Lohnes, 1 i 
l 962 . 119- z 
cussion of this phenomenon). i a hear 
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` TABLE 3 
Means, Univariate F Tests, and Scaled Vectors for Female 
Education and Nursing Students 
Means Scaled Vectors 
Needs Educ Nurs Fe p I 

Abasement 4.08 4.02 03 ns 07 
Achievement 4.56 4,62 02 ns 25 
Adaptability 5.56 5.66 .05 ns .07 
Affiliation 7.86 7.76 10 ns —.06 
Aggression 2.80 2.82 .00 ns 08 
Change 5.74 5.92 +22 ns 01 
Conjunctivity 5.20 5.08 09 ns 06 
Counteraction 5.66 6.34 3.14 07 —.21 
Deference 6.64 6.54 06 ns —.03 
Dominance 5.26 3.78 11.61 .002 .39 
Ego Achievement 4.86 3.86 4.12 04 19 
Emotionality 5.40 5.52 .10 ns 18 
Energy 6.04 6.78 4.59 03 —.29 
Exhibitionism 3.30 2.90 83 ns —.02 
Fantasized 

Achievement 2.90 2.84 02 ns —.21 
Harm Avoidance 5.58 4.78 3.80 05 +20 
Humanism 6.54 5,62 3.11 07 15 
Impulsiveness 5.62 6.00 1.10 ns —.30 
Narcissism 5.72 5.08 2.19 ns 25 
Nurturance 7.38 7.40 00 ns 00 
Objectivity 8.58 8.26 3.49 -06 +82 
Order 5.08 5.22 07 ns —.19 
Play 6.48 6.18 70 ns 09 
Practicality 5.10 5.44 64 ns 02 
Reflectiveness 6.08 6,04 .01 ns —,14 
Science 3.50 3.10 .52 ns +06 
Sensuality 5.08 5.12 01 ns —.03 
Sex 6.28 6.46 20 ns — 83 
Succorance 7.14 6,80 1.30 ns 09 
Understanding 5.36 5.54 .16 ns =.11 

*df = 1/98. 


Educ = Education 
Nurs = Nursing 


Classification 


Table 4 contains three contingency tables for predicted group 
membership (criterion groups), 4 3 X 3 table for male business, 
chemistry, and mathematics subjects, a 2 X 2 table for male bus- 
iness and mathematics students, and a 2 X 2 table for the female 
education and nursing groups. Group membership was determined 
by the maximum likelihood classification of the individual's dis- 
criminant score. This method necessitates the computation of sep- 
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arate chi squares (one for each group used in the multivariate dis- 
criminant analysis) for each subject through using the centroids 
and dispersions for each group in combination with the individual’s 
scores. Hach chi square indicates the probability of that subject's 
membership in each group employed in the discriminant analysis 
(see Cooley and Lohnes, 1962, for a discussion of this procedure). 

Since the three chi squares (Table 4) for both the male and fe- 
male groups were significant (p < .001), the hypothesis of the in- 
dependence of actual and predicted group membership was rejected. 
For the classification of the male business, chemistry, and mathe- 
matics groups, 74% of the business, 69% of the chemistry, and 
56% of the mathematics students were correctly classified. Classi- 
fication based on the second discriminant analysis, computed for 
only the business and mathematics students, resulted in the cor- 
rect classification of 90% of the business and 88% of the mathe- 
matics students. For the female groups, 82% of the education and 
84% of the nursing students were correctly classified. 

An indication of the stability of the discriminant functions was 
obtained by applying the obtained weights to the cross-validation 
groups of male and female students. Three cross-validation tables 
are summarized in Table 5. Since the chi squares for the business 
and mathematics and education and nursing students were signi- 
ficant (p = .05 and p = 02), the independence of classification 
hypothesis was again rejected. This hypothesis could not be rejected 
for the stability of the weights derived from the combined business, 
chemistry, and mathematics groups (p = .08). 

The correct classification of 65% of the education and 58% of 
the nursing students suggests that the female weights are relatively 
stable. Unfortunately, the same conclusion could not be drawn for 
the cross-validation male subjects when the discriminant analysis 
was applied to business, chemistry, and mathematics students. 
Only 54% of the business and 33% of the mathematics students 
Were correctly classified. This is accounted for, in part, by the de- 
creased probability of “hits” because of the lack of a cross-valida- 
tion chemistry group. If mathematics and chemistry students are 
considered similar (“science” as opposed to business), then they 
may be combined into one group. Following this procedure, 88% of 
the “science” students were correctly classified in the criterion 


group and 72% of the “science” students were correctly classified in 
the cross-validation sample. 
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TABLE 4 
Classification of Criterion Groups 
Predicted Group Membership 
PE era OAE CERNA 
Male Groups Female Groups 
Bus Chem Math Bus Math Educ Nurs 
Actual Bus 37 9 4 Bus 45 5 Educ 41 9 
Group Chem 5 31 9 Math 6 44 Nurs 8 42 
Membership Math 6 16 28 
x? = 177.24 x2 = 63.68 x? = 43.56 
p = <.001 p = <.001 p = <.001 
df =4 df =1 df =1 


ee a e 


Bus = Business 
Chem = Chemistry 
Math = Mathematics 
Educ = Education 
Nurs = Nursing 


TABLE 5 
Classification of Cross-validation Groups 


Predicted Group Membership 
Gees OS ATi ee SE SR 
Male Groups Female Groups 


Bus Chem Math Bus Math Educ Nurs 
Actual Group Bus 18 7 8 Bus 21 12 Educ 21 11 
Membership Math 10 14 12 Math 13 23 Nurs 16 22 

x? = 5.26 x? = 5.20 xf m= 3.82 

p = 08 p = 02 p = .05 
df =1 


df =2 df =1 


Bus = Business 
Chem = Chemistry 
Math = Mathematics 
Educ = Education 
Nurs = Nursing 


inant weights obtained from the mul- 
f business and mathematics students 


weights were relatively stable. 
of the mathematics 


Application of the discrimi 
tiple discriminant analysis 0. 
only, indicated that the derived 
Sixty-four per cent of the business and 64% 


students were correctly classified. 
Discussion 

Predictions of group membership remained above the base rate 
for both the male business and mathematics cross-validation 
groups and female education and nursing cross-validation groups. 
The findings would suggest that prediction of academic major 
based on the personality variables measured by the Activities 
Index is feasible. This observation does not exclude the fact that 
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other cognitive factors are not important components in deter- 
mining vocational goals. The inability of the discriminant weights 
to differentiate successfully between male chemistry and mathe- 
matics students would indicate that such variables are in opera- 
tion. 

Although the high mean need scores for the groups in the present 
study paralleled those needs which Stern (1962) reported as sig- 
nificantly separating similar academic groupings, these variables 
did not contribute to the classification of students in the present 
study, This was true for the education and nursing groups, particu- 
larly for needs narcissism and sex. This finding is due to the analy- 
sis utilized, namely, the multivariate approach. In such a proce- 
dure it is possible for two groups to have insignificant univariate 
F tests on a variable; yet the factor loadings may be sufficient to 
aid in classification. Application of the mulivariate technique, in 
contradistinction to the more frequently employed univariate tests, 
also takes into consideration the combined effects of a number of 
variables. Thus classification efficiency is maximized at no expense 
to the description of group dynamics. : 

The present investigation also serves to demonstrate that pre- 
diction of college career choice based on need patterns may be 
quite fruitful. Future research may adopt a similar approach in 
combining other variables with measures of personality in order to 
maximize classification efficiency and to aid the college student in 
his choice of college career plans, 
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THE RELIABILITY AND VALIDITY OF A NEW 
MEASURE OF LEVEL OF OCCUPATIONAL 
ASPIRATION?* 


BERT W. WESTBROOK 
North Carolina State University 


To meet the need for a practical, reliable, and valid measure of 
level of occupational aspiration (LOA), Haller (1957) has de- 
veloped The Occupational Aspiration Scale (OAS). Although LOA 
has been recognized as a factor of some significance in the social 
mobility of youth (Empey, 1956; Haller, 1959; Haller and Miller, 
1963; Holloway and Berreman, 1959; Rosen, 1959; Sewell, Hal- 
ler, and Straus, 1957), research has been restricted considerably 
by the lack of an instrument for measuring it. Evidence collected 
from several samples of male high school students suggests that the 
OAS is a valid measure of LOA (Miller and Haller, 1964). The 
purpose of this paper is to examine the characteristics of the OAS 
resulting from administrations of the instrument to eleventh grade 
boys and girls, 


Level of Occupational Aspiration 


Miller and Haller (1964) define LOA as the area of the occupa- 
tional prestige hierarchy which an individual views as a goal. The 
range of an individual’s LOA is determined by what he views as 
realistically probable versus idealistically desirable for him, and by 
the goals which he has for the near versus the distant future 
(Lewin, Dembo, Festinger, and Sears, 1944). Both the realistic or 
idealistic goals and the time boundaries must be specified in order 
for a subject to respond adequately to an LOA instrument. 

a This paper is partly based upon a dissertation submitted by the author in 


partial fulfillment of the requirements for the doctor’s degree at Florida State 
University, Tallahassee. 
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The Occupational Aspiration Scale 


Haller’s OAS has been described completely elsewhere (Haller 
and Miller, 1963). It is a multiple-choice instrument designed pri- 
marily for use among male high school students. Previously this 
instrument had not been employed with a sample of high school 
girls. The OAS takes into consideration idealistic and realistic lev- 
els at particular future time periods, and it combines these vari- 
bles in such a way as to yield a single score for each person. 
Questions are asked at each of two career points: short-range (S) 
and long-range (L) for the realistic level (R) and for the ideal- 
istic level (I). These four variables produce four combinations: 
RS (realistic-short), IS (idealistic-short), RL (realistic-long), 
IL (idealistic-long). The combinations are incorporated into four 
wordings for questions. The question wordings are as follows: IS— 
of the jobs listed in this question, which ONE would you choose if 
you were FREE to CHOOSE ANY of them you wished when your 
SCHOOLING IS OVER? (questions 2 and 4); IL—Of the jobs 
listed in this question, which ONE would you choose to have when 
you are 30 YEARS OLD, if you were FREE TO HAVE ANY of 
them you wished? (questions 6 and 8); RS—Of the jobs listed in 
this question, which is the BEST ONE you are REALLY SURE 
YOU CAN GET when your SCHOOLING IS OVER? (questions 1 
and 3); RL—Of the jobs listed in this question, which is the BEST 
ONE you are REALLY SURE YOU CAN HAVE by the time you 
are 30 YEARS OLD? (questions 5 and 7). Since each of the four 
questions is presented twice, there is a total of eight OAS items. 

The response alternatives for each OAS item consist of 10 oc- 
cupational titles drawn from among the 90 occupations ranked by 
the National Opinion Research Center’s study of the prestige of 
occupations (National Opinion Research Center, 1947). The ten 
least appropriate occupations were dropped, leaving a total of 
80 occupations, ten for each question. The 80 occupations were 
ranked from first to eightieth. The occupational title with the high- 
est Prestige rank was placed in the first question, the title with the 
aaa stirs placed in the second question, and so on 

: ipational title with lowest prestige rank which was 
Placed in the last question. 


Each question contains a set of occupational alternatives which 
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span the full range of occupational prestige. It is scored from zero 
to nine. Each occupation is presented only once among the eight 
items. If the respondent chooses the highest prestige occupation in 
a given question item, he receives a score of 9: if he chooses the 
lowest prestige occupation, he receives a score of 0. The sum of the 
scores on each of the eight questions may range from zero to sev- 
enty-two. The total score represents the individual’s level of oc- 
cupational aspiration. 

The OAS takes between fifteen and twenty minutes to adminis- 
ter. It may be administered easily in a group testing situation. It is 
recommended that the meaning of the occupational titles should 
not be explained to the respondents. This is necessary in order to 
reduce error due to difference among administrators and to simu- 
late the reality situation in which a student must choose among the 
alternatives he knows (Miller and Haller, 1964). 


Sample and Data 


The OAS was administerd by the investigator to a total sample 
of 154 eleventh grade students—77 boys and 87 girls during the 
spring of 1964. Additional data from this sample included schol- 
astic ability results and a questionnaire regarding future plans of 
each student. The OAS was readministered after a two-week in- 
terval and again after a five-week interval to both girls and boys. 
The distributions of total OAS raw scores for the three administra- 
tions are approximately normal. 

Table 1 shows the means, standard deviations, and ranges for 
both boys and girls on each administration of the OAS. Although 
there was no significant difference in mean OAS total scores be- 


TABLE 1 


Means, Standard Deviations, and Ranges of OAS Scores for Boys and Girls 
on the First, Second, and Third Administration 


Boys Girls 
a ee eee 
Administration Mean S.D. Range Mean S.D. Range 
First 45.78 10.84 23-07 45.47 7.61 30-60 
Second 
(Two-week Interval) 45.67 11.13 25-67 46.11 7.23 32-59 


(Five-week Interval) 45.94 12.02 23-68 44,84 8.02 28-61 
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tween boys and girls for any of the administrations, the girls’ OAS 
scores were less variable than the boys. Both the ranges and stand- 
ard deviations of the girls were smaller than those of the boys for 
each administration of the OAS, This finding can be explained by 
the fact that the occupations at the extreme ends of the OAS hi- 
erarchy are those for men primarily. Therefore, girls were forced to 
choose occupations closer to the center of the occupational prestige 
hierarchy. 
Reliability 

The reliability of the OAS was determined by computing co- 
efficients of internal consistency (Spearman-Brown) based on the 
correlation between parallel halves, and by examining the test- 
retest correlations after two-week and five-week intervals to es- 
tablish coefficients of stability. The design of the OAS makes it 
possible to determine the correlation between parallel halves since 
each half includes one complete set of question types (RS, RL, IS, 
and IL). The coefficients of internal consistency are as follows: first 
administration, 85; second administration, .83; third administra- 
tion, .86. 

The test-retest correlations provided coefficients of stability be- 
tween the same form of the test administered two weeks and five 
weeks apart. The coefficient of stability for the two-week interval 
was .84 for males and .88 for females. The coefficient of stability 
for the five-week interval was 82 for males and .78 for females. 
The standard error of measurement was computed for each ad- 
ministration of the OAS. These were found to be 6.11, 5.89, and 
5.83 for the first, second, and third administrations respectively. 
The findings show that the OAS is a fairly reliable instrument, The 
halves of the instrument appear to be equivalent, and the sta- 
bility is reasonably high over a five-weck period. 


Validity 
3 The evidence supporting the validity of the OAS comes from the 
mae structure of the instrument and from the correlation of the 
Brie scan with other measures, Level of occupational aspi- 
cate ne veo age that level of aspiration at the idealistic 
eens er than level of aspiration at the realistic level, and 
level of aspiration for long-range goals should be higher than 
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level of aspiration for short-range goals (Lewin, Dembo, Festinger, 
and Sears, 1944). In the OAS, realistic (R) questions are designed 
to tap a lower limit of the respondent’s LOA and idealistic (I) 
questions are designed to tap an upper limit of the respondent's 
LOA. Since the occupational achievement level of an individual is 
usually expected to rise to some extent during the first decade or 
so of his career, we would predict that long-range (L) LOA 
should be, on the average, higher than short-range (8) LOA. Spe- 
cific tests of these hypotheses are as follows: 

1. The realistic LOA should be less than the idealistic LOA. 
The mean OAS-RS score should be less than the mean OAS-IS 
score and the mean OAS-RL score should be less than the mean 
OAS-IL score. 

2. The short-range LOA should be lower than the long-range 
LOA. The mean OAS-RS score should be less than the mean 
OAS-RL score and the mean OAS-IS score should be less than the 
mean OAS-IL score. 

The data from the administration of the OAS partially sup- 
ported the hypotheses. The mean OAS-RS score (4.18) is less than 
the mean OAS-IS score (4.50) and the mean OAS-RL score (4.10) 
is less than the OAS-IL score (4.19). Since the mean OAS-RS score 
(4.18) is not less than the OAS-RL score (4.10), and since the 
mean OAS-IS score (4.50) is not less than the mean OAS-IL score 
(4.19), the second hypothesis was not supported. Significance tests 
were not carried out, since each subtest was judged to measure & 
different characteristic. 

To determine the factorial structure of the OAS, the eight OAS 
item scores were intercorrelated by the product-moment method 
and the resulting correlations were factor analyzed by the princi- 
pal-axes technique. Five factors accounted for 93 per cent of the 
variance; this factor matrix was rotated orthogonally by the 
Quartimax method and by the Varimax method. The Quartimax 
solution is shown in Table 2. 

Inspection of Table 2 shows that all eight OAS questions have 
moderately high loadings on Factor I. This finding tends to sup- 
port the contention that each question of the OAS contributes to 
the measurement of a general LOA variable, at least in terms of the 
factor model used. None of the loadings on Factors II and III are 
above .40. Factor I accounts for 73 per cent of the total variance 
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TABLE 2 


Quartimax Rotation of Factor Matrix of Item Scores 
on the Occupational Aspiration Scale 


Factors 

Question-Item I I I IV v 
ee mr OS AE A ce 
1. Realistic-Short Range 54 05 09 42 07 
2. Idealistic-Short Range 59 —30 —34 -10 28 
8. Realistic-Short Range 71 05 —05 26 11 
4, Idealistic-Short Range 63 -11 -19 09 19 
5, Realistic~Long Range 68 30 11 -13 —04 
6. Idealistic-Long Range 64 02 24 —27 00 
7. Realistic-Long Range 61 26 08 03 09 
8. Idealistic—Long Range 66 —20 16 —04 -13 
o L US STO erg OE EEE SAR CRD e 

Cumulative Proportion 
of Total Variance 73 80 86 90 93 


Note,—Alll decimal points omitted. 


and might be referred to as general level of occupational aspiration. 

Theoretically, the nature of level of aspiration leads one to ex- 
pect that group factors would emerge from items in the OAS. The 
group factors might be called realistic-short, idealistic-short, re- 
alistic-long, and idealistic-long. In an attempt to obtain a factor 
solution which would account for the hypothesized group factors, 
the original factor matrix was rotated by the Varimax method. It 
can be seen in Table 3 that the Varimax solution accounts for the 
group factors to a greater extent than the Quartimax solution did.? 
The two highest loadings on each factor are found on those ques- 
tions which presumably measure identical dimensions of aspira- 
tion level, The two questions with the highest loadings on Factor 
I are questions dealing with Tealistic-long range goals; these are 
questions five and seven. The two idealistic-long range questions, 
questions six and eight, are highest on Factor IIT. Factor IV loads 
highest on questions one and three; both questions pertain to re- 
alistic-short Tange occupational choices. Although Factor V does 
not contain an appreciable loading for any of the questions in the 
OAS, the questions pertaining to idealistic-short range goals were 

*Tn a personal communicati investi i i 

methode of faciar analysis De. Willan B, Miched usenet he Vane 


timax method. This suggestion i actory s y 
a Produced a factor genet ead appreciated, since the Varimax 


esized structure of occupational aspiration level. 
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TABLE 3 


Varimazx Rotation of Factor Matrix of Item Scores 
on the Occupational Aspiration Scale 


Factors 
PA N a OR iS AN A 
ios. CASES ID. Se Sa ee 
1. Realistic-Short Range 49 08 -01 43 05 
2. Idealistic-Short Range 53 -18 —21 —06 36 
3. Realistic-Short Range 61 04 00 38 14 
4. Idealistic-Short Range 55 -10 -17 13 29 
5. Realistic-Long Range 57 44 10 —05 —07 
6. Idealistic-Long Range 50 12 33 —16 12 
7. Realistic-Long Range 51 42 07 -iL 15 
8. Idealistic~Long Range 54 -17 28 08 =11 
Cumulative Proportion 
of Total Variance 73 80 86 90 93 


Note.—All decimal points omitted. 


found to have the highest loadings. Many of the factor loadings do 
not reach the .40 level; nevertheless, the pattern of loadings is 
consistent with the hypothesized structure suggested by level of 
aspiration theory and research. 


Relationship between OAS and Other Variables 


Additional evidence of the validity of the OAS was collected by 
examining the correlation of the total score with another instru- 
ment presumed to be a measure of level of occupational aspiration. 
An open-ended measure of LOA, known as the North-Hatt tech- 
nique, was administered to all pupils in the present study. Miller 
and Haller (1964) found this instrument to be correlated with level 
of educational and occupational attainment several years later (.46) 
and with number of years completed at college (.52). The respond- 
ent is asked to indicate the occupations he would enter if he were 
absolutely free to go into any kind of work he wanted, the occupa- 
tions he has thought about going into, the occupation he plans to 
follow, and the type of work he would like to be doing by the time 
he is 30 years old. The average prestige scores of the responses to 
these questions have a meaning similar to that of the OAS. When 
the two sets of scores obtained in this study were correlated, it was 
found that the two instruments correlated to the extent of .68. 

A number of additional variables were found to be correlated 
with OAS scores. The correlations were as follows: number of years 
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of college planned (.69); high school grade-point average (.57); 
scholastic ability-SCAT (.45) ; parental desire for respondent’s edu- 
cational achievement (.48); and socio-economic status of the re- 
spondent’s family (.43). 


Summary 


The evidence collected in this study strongly suggests that the 
OAS is a reliable and valid measure of level of occupational as- 
piration. Test-retest reliability coefficients over a period of two 
weeks and five weeks were found to be satisfactory. Hypotheses 
about the elevation of mean OAS item scores were partially sup- 
ported. Although boys and girls did not differ significantly on the 
OAS total scores, the girls’ scores showed less variability. 

The factorial structure of the OAS derived from a Varimax ro- 
tation of the factor matrix supports the existence of both a general 
LOA factor and a number of group factors suggested by level of 
aspiration theory and previous research. The OAS should continue 
to be a useful instrument for studying level of occupational aspira- 
tion among high school pupils. 
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FURTHER VALIDATION OF A SCALE TO MEASURE 
PHILOSOPHIC-MINDEDNESS 


DONALD W. FELKER 


Institute for Child Study 
University of Maryland 


Tus study is concerned with validation of the PM Scale which 
is designed to measure philosophic-mindedness (PM). It was orig- 
inally presented by Felker and Smith (1966). 

The PM Scale has 84 items, in three sections. Hach section is 
composed of 28 items; 14 deal with educational material and 14 
deal with general material. The items are all forced-choice with two 
alternatives. 

Section one of the scale presents four paragraphs describing sit- 
uations in which teachers might find themselves. After each para- 
graph the testee is presented with a series of paired statements. 
He is asked to choose the statement in each pair describing what he 
would be more likely to do if faced by the situation described in the 
paragraph. 

Section two asks the testee to choose the statement in each pair 
which describes more accurately a person whose judgment he would 
respect. 

Section three asks the testee to choose the statement in each pair 
with which he more closely agrees. 

Analysis of philosophic-mindedness was begun by Smith (1956). 
A person is philosophic-minded whose thinking is characterized by 
three dimensions, i.e., comprehensiveness, penetration, and flexibil- 
ity. The PM Scale was developed using flexibility as its criterion 
attribute. Philosophic-minded flexibility was postulated as having 
four characteristics. These characteristics are: 
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1. Being free from psychological rigidity. 

2. Ability to evaluate ideas apart from their source. 

8. Seeing issues as many-sided rather than two-sided, and the 
development of a relatively large number of alternate hypotheses, 
explanations, or viewpoints. 

4, Maintaining a tolerance for tentativeness and suspended judg- 
ment and a willingness to take action in an ambiguous situation. 

The PM Scale yields a total score and four subscores. The sub- 
scores are individual measures of these four characteristics. For the 
present study the total score appeared to be the most relevant, and 
hence the analysis of the subscores is not reported. 


Review of Previous Study 


The original study involving the scale dealt with the construc- 
tion of items, the general characteristics of the scale, and hypothe- 
ses which were tested to begin construct validation of the scale. 

When the test was administered to 358 subjects enrolled in grad- 
uate courses in education, 76 items were correlated at the .01 level 
of significance with the criterion scores, and two items were cor- 
related at the .05 level of significance with the criterion scores. The 
remaining six items had positive correlations which were not sig- 
nificant. 

The PM Scale exhibited a test-retest reliability coefficient of 
.80 with approximately three months between testings. The Spear- 
man-Brown split half reliability coefficient was also .80. 

The hypotheses which were tested in the previous study to begin 
construct validation of the scale and the results were: 

1. If the scale measures PM, it will significantly separate stu- 
dents who are rated on flexibility by their instructors in philosophy 
of education courses. Those rated high on flexibility scored signif- 
icantly higher on the PM Scale than those rated low. The differ- 
ences were significant at the .001 level. 

2. If the scale measures PM the scores will have a negative re- 
lationship to dogmatism and closed-mindedness. The correlation 
between the PM Scale and Rokeach’s Dogmatism Scale was —.393. 


The analysis of the findings of this study tended to support the 
scale as being a valid measure of PM. 
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Purpose 


The purpose of the present study was to continue the investiga- 
tion of the validity of this scale. Three hypotheses were presented 
and tested. These hypotheses and their relationship to the con- 
struct are: 

Hypothesis One: If the test measures PM, there should be an 
increase in scores subsequent to taking courses in philosophy of ed- 
ucation that have this attribute as their aim. 

Although Felker and Smith (1966) presented this hypothesis, it 
was not tested. PM is an attribute that is changeable by learning 
experiences. There should be a general increase in scores associated 
with experiences designed to increase the attribute. The present 
study tested this hypothesis by using a group before and after it 
had taken an introductory course in philosophy of education. 

Hypothesis Two: If the scale measures PM, there should be a 
significant positive correlation between the pretest PM score and 
success in the course as measured by course grades. 

Since “Introductory Philosophy of Education” deals with ques- 
tions which are judged to require the types of skills and intellectual 
dispositions which are characteristic of PM, it would follow that 
the person who possesses a greater measure of these characteristics 
should be better prepared for the course. Although these students 
should be better prepared, one would predict only a moderate cor- 
relation because of the changes that would take place over the 
course and because of the limitation posed by the reliability of the 
criterion. 

Hypothesis Three: If the scale measures PM, there should be a 
significant positive correlation between PM scores on the posttest 
and success in the course as measured by course grades. 

The grades in the course were computed from two types of assign- 
ments. Each student was given two objective tests, and two papers 
(one on a subject of the student's choice, the other on an assigned 
topic). Both papers were assigned with the purpose of displaying 
the characteristics of PM. The student was graded on the basis of 
the degree of PM displayed in attacking the problem of the paper. 

The aim of this course was the cultivation of PM, and the grades 
in the course were meant to be a measure of the attainment of this 
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goal. In both the course grades and the PM Scale, one should have 
two experimentally independent measures of PM for which a posi- 
tive correlation should be observed. 


Procedures 


Two classes at Indiana University in “Introductory Philosophy 
of Education” were each randomly divided into two groups. Two of 
these groups were given the PM Scale as a pretest. One of the 
groups was released from the class time when the PM Scale was 
administered. The fourth group wrote a brief essay on the subject 
“How will philosophy of education help me in my work?” while 
the pretest was being given to the rest of the class. All of the sub- ` 
jects were given the PM Scale at the end of the course. 

These procedures were followed in order to test Hypothesis One. 
Two alternate explanations might be given for an increase in scores 
from the pretest to the posttest. These alternate explanations would ` 
be: (a) The increase is due to taking the test a second time, or (b) 
The pretest would make the student more aware of what was going 
to be presented in the course and hence would interact with the 
course to produce an increase in scores over the course. 

A group of 103 students who were not enrolled in a philosophy of 
education course showed no increase in scores from a first to a sec- 
ond administration. This would make the first alternate explana- 
tion unacceptable. 

In order to test the second explanation an analysis of variance 
was computed for the posttest scores of the four experimental 
groups. If gains in scores could be explained by the taking of the 
pretest and then the course, a significant difference between those 
who had taken the pretest and those who had not taken the pretest 
should have appeared on the posttest. The analysis of variance of 
the posttest scores showed no significant difference between the 
groups. 

For the testing of the three validation hypotheses which were 
Proposed in the study the subjects were divided into two groups; 
those who had taken the pretest and those who had not taken the 
pretest. There were 47 subjects in each group. 

The purpose of the pre- and posttests of PM was not explained 
to the subjects until all had taken the posttest. The PM Scale was 
not scored until all course work was completed and the grades cal- 
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culated. The course tests and papers were graded on a nine-point 
scale, A+ to C—. The total point value of the two course tests and 
the two course papers was then used in computing the correlations 
in this study. 

Since all members of the experimental group were under the same 
instructor in “Introductory Philosophy of Education,” a second 
group under a different instructor was used for cross-validation 
purposes. There were 31 subjects in this group. 


Results 


Hypothesis One: If the test measures PM, there should be an 
increase in scores associated with taking courses in philosophy of 
education that have this attribute as their aim. 


TABLE 1 


Repeated Measures Analysis of Variance of PM Scores on Pretest and Posttest 
for Experimental Group and Cross-V alidation Group 


Group Source df ss MS F Sign. Level 
Experimental Between 1 1744.95 1744.95 49.08 001 
Residual 46 1635.55 35.56 
Cross- 
Validation Between 1 248.00 248.00 10.81 01 


Residual 30 688.00 22.93 

An analysis of variance was computed for PM scores on the pre- 
test and the posttest for the group enrolled in philosophy of edu- 
cation. This analysis is presented in Table 1. Table 1 also pre- 
sents the analyses of variance for pretest and posttest PM scores 
for the cross-validation group. A repeated measures analysis was 
used for both of these analyses of variance. Both of these groups 
showed significant gains in PM scores subsequent to the course. 
These analyses would tend to support the original hypothesis. 

Hypothesis Two: If the scale measures PM there should be a 
significant positive correlation between the pretest PM score and 
course grades. The PM Scale and the course grades for the pretest 
group had a Pearson correlation of .507. For the cross-validation 
group, the correlation of the same variables was .506. These cor- 
relations were significant beyond the .01 level and would support 
the hypothesis. 

Hypothesis Three: If the scale measures PM there should be a 
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significant positive correlation between the posttest PM score and 
the course grades. 

This hypothesis was tested for three groups: those who had taken 
a PM pretest, those who had not, and the cross-validation group. 
It was not known whether the taking of the pretest would be re- 
lated to any significant differences in the correlations. The correla- 
tion between PM and grades in the group that had taken the pre- 
test was .559. The correlation between PM and the grades in the 
group that had not taken the pretest was .767. These two cor- 
relations are not significantly different at the .05 level. Both of the 
correlations, however, are significant beyond the .01 level. 

The correlation between PM and grades in the cross-validation 
group was .518 which is significant beyond the .01 level. 

These correlations would support the hypothesis. 


Discussion and Summary 


Three hypotheses were proposed in the process of construct vali- 
dation of the PM Scale. The hypotheses were tested using 94 grad- 
uate students enrolled in a beginning course in philosophy of ed- 
ucation. This course, which had the cultivation of PM as its aim, 
would seem to be suited to testing the hypotheses. 

The hypotheses were also tested on another class of 31 subjects 
for cross-validation purposes. 

Each of the hypotheses received positive support from the analy- 
ses. Differences between pre- and post-administration of the PM 
Scale were significant at the .001 level. The PM scores, both the 
pretest and the posttest, were significantly correlated with course 
grades, 

Although no measure of general intelligence was obtained on the 
subjects in this study, a previous study by Felker and Smith (1966) 
showed a correlation of 421 between a general measure of intelli- 
gence and the PM Scale. Although this correlation is significant, 
it accounts for only a small portion of the variance in the PM 
Scale. Because of this it would not appear that the results of our 
present hypotheses would be adequately explained on the basis of 
general intelligence. 

The hypotheses tested in this study and the results observed 
would continue to support the PM Scale as a valid measure of the 
stated attribute of philosophic-mindedness. 
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THE RELATIONSHIP OF THE 1960 REVISED 
STANFORD-BINET INTELLIGENCE SCALE TO 
INTELLIGENCE AND ACHIEVEMENT TEST SCORES 
OVER A THREE-YEAR PERIOD 


WILLIAM D. CHURCHILL ann STUART E. SMITH 
Alfred University 


Aw reviews of the third revision of the Stanford-Binet Intelli- 
gence Scale (1960 S-B) essentially agree that the latest revision 
represents an advance in technical quality over the 1937 edition. 
However, it was noted by most of the reviewers that its predictive 
validity had not been ascertained at the time of publication, Bal- 
insky (1960), pp. 155-156) noted that “the probability is high that 
the validity of the revision will be at least equal to if not greater 
than the 1937 version” because of the great amount of overlap be- 
tween the two revisions. Fraser’s review in Buros (1965, pp. 830- 
831) indicates that “there is no guarantee that the final test, taken 
as a whole, will give the same results as either the old Form L or 
Form M.” Subsequent to 1960, only a few validity studies have 
been published. For a sample of elementary school children, Estes, 
et al. (1961) reported a correlation of .82 between the 1960 S-B and 
the 1937 S-B. In this same study a correlation of .74 between the 
1960 S-B and the WISC (full scale) was reported. Tatham and 
Dole (1963) reported a correlation of .41 between the 1960 S-B and 
the California Test of Mental Maturity-SF for one group of ele- 
mentary students; for a second group they reported a correlation 
of .56. 


Purpose 


The primary objective of this study was to determine the re- 
lationship between the 1960 S-B and the Lorge-Thorndike IQs ob- 


1015 


1016 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


tained three years later. A secondary objective was to determine 
the relationship between the 1960 S-B and achievement as meas- 
ured by a standardized achievement test battery. 


Method 


At the time the criterion data were collected for this study, the 
subjects were members of the seventh grade in a rural central 
school of New York State. When these students were in the third 
grade, the 1960 S-B was administered to each member of the class. 
There were 82 members in the class, 45 boys and 37 girls. Because 
of attrition over a three-year period, the authors were able to com- 
pile complete data on 56 members of this group, 32 boys and 24 
girls, 

The data were obtained from the school’s records of test infor- 
mation on file for each subject in the sample, as follows: 

1. IQ, 1960 S-B, administered grade 3 

2. Composite score, Iowa Test of Basic Skills, Form 2, 1955 
edition, administered grade 3, fall semester 

8. Composite score, Iowa Test of Basic Skills, Form 1, 1955 
edition, administered grade 6, fall semester 

4. IQ, Lorge-Thorndike Intelligence Tests, Verbal and Non- 
verbal batteries, Form A Level 4, administered grade 6, fall 
semester 

The third grade 1960 S-B IQs were correlated with the sixth 
grade Lorge-Thorndike verbal IQ (L-T verbal) and the Lorge- 
Thorndike nonverbal IQ (L-T nonverbal). Correlations were ob- 
tained also between the 1960 S-B, L-T verbal, L-T nonverbal, and 
the Iowa Test of Basic Skills (ITBS) achievement scores (com- 
posite) obtained in grade three and grade six. The intercorrela- 
tions among the tests are shown in Table 1, together with the re- 
spective means and standard deviations. Tabulations of changes 
in scores between the 1960 S-B and the L-T verbal and L-T non- 
verbal were accomplished by inspection, with the results as indi- 
cated in Tables 2 and 3. 


Results 


The Mean 1960 S-B for this sample of 56 students was 110.25 
compared with an L-T verbal mean of 110.92. The obtained standard 
deviation for the 1960 S-B is 15.8 which corresponds closely to the 
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standard deviation of 16.0 reported by Terman and Merrill (1960, 
p. 28). The L-T standard deviations are somewhat smaller (12.5 
and 14.1) than the reported value of 16 (Lorge and Thorndike, 
1954, p. 3). 


TABLE 1 


Means, Standard Deviations, and Intercorrelations for 1960 Stanford-Binet, 
Lorge-Thorndike, and Iowa Tests of Basic Skills 


L-T L-T ITBS ITBS 
Test x SD S-B N. NV 3 6 
S-B 110.25 15.89 — 79 65 59 +74 
L-T (V) 110.92 12.52 =, 61 62 84 
L-T (NV) 113.44 14.19 vont 47 65 
ITBS (3) 3.50 -17 = +79 
ITBS (6) 6.59 1.24 = 


The primary objective of this study was to determine the predic- 
tive validity of the 1960 S-B over a three-year period using the L-T 
IQs as criteria. For the 56 students, the obtained correlation be- 
tween the third grade 1960 S-B and the L-T verbal was .79, and the 
correlation between the third grade 1960 S-B and L-T nonverbal 
was .65. 

Table 2 shows the amount of change in IQs on the L-T verbal 
compared with the 1960 S-B. Exactly one-half of the sample, 28 
students (including one who registered no change), obtained IQs in 
the range + 5 points of their third grade 1960 S-B IQs. Three stu- 
dents obtained L-T verbal IQs which were 16 to 25 points above. 
One student showed no change in IQ between the 1960 S-B and the 
L-T verbal. (This person’s score has been omitted in Table 2.) 

Table 3 shows the amount of change in the L-T nonverbal IQs 
compared with the third grade 1960 S-B IQs. Eighteen students 
(including two who showed no change) obtained L-T nonverbal 


TABLE 2 


Increase or Decrease in L-T Verbal IQs from 1960 S-B IQs 
for 55 Students over a Three-year Period 


Score change 15 610 11-15 16-20 21-25 26-30 Total 
TETA 2 8 5 1 2 0 28 
‘Deareass 15 OB 4 2 0 1 27 
Total 2 18 9 3 2 1 55/55 
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1Qs in the range of + 5 points of their third grade S-B IQs. One stu- 
dent’s IQ was 30 points higher than his third grade 1960 S-B IQ, 
while one student’s L-T nonverbal IQ was 36 points lower than his 
third grade 1960 S-B IQ. A majority, 34 students, scored higher 
on the L-T nonverbal than they did on the 1960 S-B. This finding 
reflects the higher L-T nonverbal mean of 113.44 as compared with 
the 1960 S-B mean of 110.25. Two students showed no change be- 
tween the 1960 S-B and the L-T nonverbal. (These two students 
have been omitted from Table 3.) 

The correlation between the 1960 S-B and the third grade ITBS 


is .59, the correlation between the 1960 S-B and the sixth grade 
ITBS is higher, .74, 


TABLE 3 


Increase or Decrease in L-T Nonverbal IQs from 1960 S-B I Qs 
for 55 Students over a Three-year Period 


Score 

change 1-5 6-10 11-15 16-20 21-25 26-30 31-35 36-40 Total 
Increase iis 9 6 2 1 34 
Decrease TASS 1 2 1 20 


1 

Total LOIS O MNT wy “154/54 
SS a AE L, 

The intercorrelations among the four criterion variables are pre- 
sented in Table 1. For this sample, the intercorrelation between 
the two parts of the Lorge-Thorndike is 62. The highest obtained 
84) is between the L-T verbal and the 
gh it is only slightly higher than the next 


Discussion 


The correlations obtained in this study between the 1960 S-B 
and the L-T verbal was .79. This correlation, based upon two dif- 
ferent tests over a thre i 
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tion of .82 has been reported (Krugman et al., 1951) between the 
1937 S-B (form L) and the WISC (full scale). The correlation of 
the 1937 S-B (form L) with the WISC verbal scale was .74, and 
with the WISC performance scale the correlation was .64. It might 
be noted, however, that these correlations were based on scores 
obtained over a period of only a few months. It seems reasonable 
to assume that the correlations between the 1937 S-B and the 
WISC over a three-year period would be somewhat lower than those 
reported in the Krugman study. 

The correlation between the 1960 S-B and the L-T nonverbal is 
.65, somewhat lower than the correlation of .79 between the 1960 
S-B and the L-T verbal. This parallels the findings reported by 
Krugman et al., in which the WISC performance IQs correlated 
.64 with the 1937 S-B (form L). 

The correlation of the 1960 S-B with the ITBS shows obtained 
correlations of .59 for grade three and .74 for grade six. It is un- 
usual to find concurrent validity—third grade vs. third grade—to 
be lower than predictive validity—third grade vs. sixth grade. 

Inasmuch as the four separate correlations involving the 1960 
S-B and the four tests of “scholastic ability” correlate between .59 
and .79, it appears that the 1960 S-B is a valid measure of “scholas- 
tic ability.” Cronbach (1960, pp. 224-225) points out that “for 
most children, a group mental test leads to the same prediction that 
a comprehensive achievement test would.” Referring to Table 1, it is 
noted that the correlation between the L-T verbal and the sixth 
grade ITBS is high (.84); the correlation of the L-T nonverbal and 
the sixth grade ITBS, while somewhat lower, is still quite high 
(.65). The correlation between the 1960 S-B and the sixth grade 
ITBS (.74) is of the same relatively high order as the correlation 
between the third grade ITBS and the sixth grade ITBS (.79). 
These results would seem to indicate that, for this sample, an in- 
dividual mental test, a group mental test, and a comprehensive 
achievement battery are comparable in their predictive ability. 
These findings support Cronbach’s generalization that group mental 
tests and comprehensive achievement tests lead to the same predic- 
tion. 

Correlation coefficients are typically used to express or index the 
validity of intelligence tests. Another useful way to show the rela- 
tionship between two tests is to show what range of IQs exist on a 
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second test for a given IQ interval on the first test. Although an — 
expectancy table is not presented, the data in Table 2 may be help- 
ful in this respect. One-half of the students obtained L-T verbal ` 
IQs which were not more than + 5 points of their 1960 S-B IQs ob- 
tained three years earlier, Although the variability of the L-T ver- 
bal score at grade six is somewhat less than the variability of the 
1960 S-B scores at grade three, the mean scores are practically iden- ~ 
tical. Based upon the data for this small sample, it appears that the 
two IQs can be used practically interchangeably. 

Although the sample used in this study was small, the results 
presented here indicate that the latest revision of the Stanford- 
Binet is as useful a predictor of scholastic achievement as the 1937 
revision. Further studies, involving larger and more diverse sam- 
ples, are needed to establish the predictive validity of the 1960 S-B. 
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PREDICTION OF THE EMPLOYABILITY OF STUDENTS IN 
A SPECIAL EDUCATION WORK-TRAINING PROGRAM 
USING THE PORTEUS MAZE TEST AND A RATING SCALE 
OF PERSONAL EFFECTIVENESS! 


SALVATORE GAMBARO anp ROBERT E. SCHELL 
Michigan State University 


Tum present investigation is best described as an empirical pre- 
diction study. It was concerned with the use of the Porteus Maze 
test as a predictor of employability among educationally handi- 
capped adolescents. The question prompting the study may be 
phrased in this form: Can a dependable cutting score be derived 
on the Porteus Maze test that will separate those individuals who 
will later be hired from those who will not? 

The predictive relationship between rated personal characteris- 
tics and employability was also investigated. Comparison of this 
relationship with that of the Porteus test and employability made it 
possible to evaluate whether the better selection procedure would 
have been to use the Porteus test alone or in combination with 
another instrument. 


Method 


Subjects 


The Ss were 71 adolescents—25 girls and 46 boys—between 16 
and 19 years of age. This sample constituted all of those from a 
Lansing Special Education Department population of 84, for whom 
complete and usable test information was obtained.? All of the Ss 


1 Thanks are due Dr. Marvin Kaplan, Director of Psychological Services, 
Lansing, Michigan, for his support and assistance. 

2 At the beginning of the study there were 62 different employers and 84 
possible Ss. Thirteen of the possible 84 Ss could not be included in the study: 
2 dropped out of the program and could not be tested, 3 had unusable rating 
sheets because of misunderstood instructions, and 8 employers failed to return 
rating sheets. 
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were in a work-training program for the educationally handicapped 
in operation at the high school level in the Lansing, Michigan, 
Public Schools. Students in this program go to school on a part- 
time basis and work part time for employers cooperating in the 
program. The Special Education Department maintains classes in 
three high schools in the city of Lansing, and every S attends that 
school which is geographically closest to his home. Each S in 
the program had obtained a WISC, WAIS, or Stanford-Binet 1Q 
score between 50 and 90, had a history of course work failure for 
the last two or more years, and showed retardation of two or more 
years on scholastic achievement tests. Participation in the program 
was initially recommended by the appropriate school authority, re- 
quired the concurrence of the parent(s), and was voluntarily en- 
tered into by the student. 

Cutting Score (CS) and Replication (R) Samples. The 40 Ss 
who were in the work-training program during the first year it was 
fully instituted formed the CS sample. The 31 Ss who were in the 
work-training program during its second year of existence formed 
the R sample. 


Measures 


Porteus Maze Test (PMT). The PMT was administered and 
scored for each S in accordance with the instructions set forth by 
Porteus (1959). The S’s derived test age score was used for pur- 
Poses of analysis. Following Porteus, this score was based on the 
total number of mazes successfully completed, taking into con- 
sideration the number of trials required. 

$ Rating Scale of Personal Effectiveness (RSPE). Personal effec- 

veness was assessed by a rating scale devised by Warren (1961) 
for use with mentally retarded adolescents. The scale is made up of 
18 items, the items having to do with such characteristics as punc- 
tuality, cooperativeness, and showing initiative. 

A RSPE was filled out on each S by both his teacher and em- 
ployer. Neither evaluator knew that the other one was also filling 
out a rating scale on the S, The teacher was asked to rate the S on 


ee ERA AE 
2 The relationship between scores on the PMT i 
Neer 2 d each item on the RSPE 
was determined in a ilot 8 hist FA $ F; 
tween the PMT and io of cae bay oe Positive relationship be- 


ms, a low positive relationship between the 


TAAT and 5 of the items, and no relationship beter the PMT and 3 of the 
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each item in relation to his average student in the program, and 
the employer in relation to his average employee. For each item the 
S was to be rated as “less than,” “same as,” or “more than” the 
average student or employee.* 

In order to avoid obvious contamination of the criterion measure, 
only the teachers’ ratings were scored for use in data analysis. The 
RSPE was scored to yield a total weighted rating score for each S. 
For any item, “less than” ratings were assigned a weighted score of 
1, “same as” a score of 2, and “more than” a score of 3. The mini- 
mum possible score was 18 and the maximum 54. 


Employability 

At the bottom of the RSPE sheet, employers checked one of four 
categories—Yes, Probably, Probably Not, No—in response to the 
question, “Would you be willing to hire this individual as you 
would your average applicant, if a job were available?” This em- 
ployability item was the criterion measure. Those Ss for whom the 
employer checked either “Yes” or “Probably” were classed together 
under Employable. Those Ss for whom the employer checked either 
“Probably Not” or “No” were classed together under Not Em- 
ployable. In the case of the R sample, it was also possible to ob- 
tain the additional criterion measure of who actually did or did not 
get hired later by the employers. 


Procedure 

The Porteus test was administered to each S in a private office of 
the school he was attending. 

At the head of each group of Special Education students in the 
three high schools is a teacher. Besides teaching he has a regular 
time set aside each day for counseling and placement. He obtains 
work-training employment for the students with an employer will- 
ing to cooperate in the program, and maintains close communication 
with the employer in order to gauge the students’ work progress. 
The three teachers were asked to fill out a RSPE on each of their 
student-trainees. They were given a form letter which described 
the project and gave instructions on how to fill in the rating sheet. 
In the form letter they were informed that their responses would be 
used in a research project and they were encouraged to be as accu- 


4In the pilot study the teacher and employer ratings correlated .94. 
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rate as possible. They were also reassured that their ratings would 
in no way influence the S's school or job standing. 

Every employer was sent a rating sheet with his traince’s name _ 
on it, a copy of another form letter, and a stamped addressed en- 
velope for the return of the rating sheet. The form letter sent to the 
employer was the same as that given to the teacher-coordinator _ 
except for slight changes of phrasing to make it more appropri- 
ate to the work situation. 

The steps in data collection were the same for all Ss. During the ; 
time the rating scales were being filled out and returned by the 
teachers and employers, the PMT was administered and scored for 
all the Ss. After all the rating sheets were returned, or otherwise | 
accounted for, they were scored. The scored protocols for both the 
PMT and RSPE were then identified as belonging to a particular 
S in either the CS or R groups. The data were subsequently as- 
sembled and analyzed separately for first the CS and then the R — 
sample. 


Results 


There were no sex or age differences on either the PMT or 
RSPE. Therefore, sex and age were disregarded in all analyses. 
Fisher exact probability tests and one-tailed rejection regions were 
used for all comparisons, The p values reported are exact prob- 
ais The entries in the cells in Tables 1 and 2 are number of 

S. 

The results for the CS sample are most conveniently considered 
first; then the results for the R sample. 


CS Sample 


PM T. A test age cutting score of 13.7 years on the PMT gave 
maximum success in discrimination. Part A of Table 1 presents the 


of the Ss in this sample, 
occur for 100 per cent of 
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gory; or, in other words, the proportion of false negatives within 
the Employable category is less than the proportion of false posi- 
tives within the Not Employable category. 

RSPE. A cutting score of 33.5 on the RSPE was maximally ef- 
fective in discriminating those who were likely to be hired or not. 
The probability associated with a set of observations as or more 
extreme than those in Part B of Table 1 is .0005. Correct placement 
is again impressively frequent; 35 out of 40 Ss are correctly clas- 
sified. In comparison to the corresponding PMT data, the propor- 
tion of false negatives is about the same while the proportion of 
false positives drops a bit. 

PMT and RSPE. A brief re-examination of Parts A and B 
shows that better prediction is obtained for those Ss scoring above 
than for those scoring below the cutting score on either the PMT 
or RSPE. Both score distributions also show some negative skew- 
ing, although this is not apparent from the data presented. In Part 
C of Table 1 the data have been assembled using the S’s score 
position on both the PMT and RSPE. 

Inspection shows that using this double standard results in per- 
fect placement of those Ss scoring above or below the cutting score 
on both scales. It eliminates both the false positives and false nega- 
tives. A Fisher test based on the data presented in the upper por- 
tion of Part © gives a p value of .00001. The set of differences 
observed in the lower portion is not statistically significant, indi- 


TABLE 1 
Comparison of Score Position and Employer's Judgment for CS Subjects 
Judgment of Employer 
Score* Not 
Part Test Position Employable Employable 
Plus 26 3 
A PMT Minus 5 6 
Ph 27 1 
B RSPE Minus 4 7 
Plus-Plus 22 0 
Minus-Minus 0 5 
Cc PMT-RSPE i 
Plus-Minus 4 3 
Minus-Plus 5 1 


a Plus indicates above the cutting score, minus below. 
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cating that scoring above the cutting value on the RSPE and 
below on the PMT is not significantly more likely to result in 
being considered employable than scoring above on the PMT and 
below on the RSPE. 


R Sample 


PMT. Having established cutting scores on the PMT and RSPE 
with the CS sample, the next question was whether or not these 
Scores would continue to be as predictive with a new sample of 


The cutting score on the PMT derived with the CS group con- 
tinued to give a high rate of correct prediction, Inspection of Part A 
of Table 2 shows that correct prediction was obtained for 23 of the 
31 Ss, and this overall hit-rate is comparable to that observed in 
the CS sample. A Fisher test on the set of observations in Part A 


category hit-rate differences for the CS and R samples shows that 
they do not differ significantly, 

RSPE. The original cutting value on the RSPE also continued 
to yield a high rate of correct prediction. The probability associated 


TABLE 2 

Comparison of Score Position and Employer's Judgment for R Subjects 

Judgment of Employer 

Scores Not 

Part Test Position Employable Employable 

A P Plus 15 4 
eS Minus 4 8 
B RSPE Plus 16 0 
Minus 3 12 
Plus-Plus 13 0 
C  - PMT-RSPE FEES z Š 
Plus-Minus 2 4 
0 


i 
= 
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with a set of observations as or more extreme than those in Part 
B is .000003. Twenty-eight of the total 31 Ss are correctly placed. 
There are no false positives, and there are three false negatives. The 
overall and per-category hit-rates observed here differ very little 
from those observed with the CS sample (see Table 1, Part B). 

Somewhat higher overall and per-category hit-rates with the 
RSPE than with the PMT again occur with the R sample. However, 
the overall and—except in one instance—the per-category hit-rate 
differences of the two scales are not statistically significant for 
either the CS or the R sample. The one exception occurs with the 
R sample where the hit rate in the Employable category is sig- 
nificantly higher using the RSPE (p < .01, by binomial expansion). 

PMT and RSPE. Further inspection of the data indicated that, 
as with the CS sample, better prediction is obtained in those in- 
stances where a S scores above or below the critical value on 
both scales. In Part C of Table 2 the data have been assembled 
using the S’s score position on both the PMT and RSPE. The 
probability associated with a set of observed values as or more 
extreme than those in the upper portion of Part © is .00003. Perfect 
prediction is again obtained for those Ss scoring above the critical 
values. Except for one false negative, this would also be the case 
for those Ss scoring below the critical values. 

As with the CS sample it also appears from inspection that Ss 
scoring above the cutting score on the RSPE and below on the PMT 
are more likely to be judged hirable by the employer. However, the 
set of differences presented in the lower portion of Table 2 are not 
significant. In addition, if the data from both samples are combined, 
the differences are still not significant. 

Finally, as previously noted, it was possible to find out who 
among the 31 Ss in the R sample eventually got hired and who did 
not. Of the 12 Ss classified as Not Employable, none was hired 
later by his respective employer. Of the 19 Ss classified as Employ- 
able, 18 were hired later by their respective employers. The remain- 
ing S would have been hired, but his family had moved out of the 
state. 

Discussion 

As Special Education Departments continue to expand their 

school work-training programs, and as the number of potential em- 
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ployers who can be enlisted to cooperate in such programs ap- 
proaches an asymptote, more careful screening of those to be trained 
will become increasingly necessary. Given this eventuality, the 
Porteus Maze test could prove to be a helpful screening device, 
Apart from the fact that it is relatively inexpensive as well as sim- 
ple and quick to administer, the present results suggest that it might 
profitably be used as an initial screening instrument, If, for example, 
in the present study only those Ss scoring above the cutting score 
had been admitted into the training program, 85 per cent of them 
would presumably have been hired by their employers. 

Should this relatively high rate of initial screening be judged not 
high enough, however, the results for the rating scale indicate that 
if it is used in combination with the Porteus Maze test, even further 
successful screening might be Possible. If, for instance, only those 
Ss who scored above the critical value on both scales had been ad- 
mitted into the training program, 100 per cent of them would pre- 
sumably have been hired by their employers, 

Data were not collected that would allow some kind of quantifica- 
tion of savings—in terms of cost, time, or efficiency of training in the 
Program—to be derived by screening out those who would later 
Prove to be unhirable. It does seem likely, however, that an ap- 
preciable and worthwhile savings could accrue by instituting both 
Scales as screening procedures. The Porteus Maze test could be used 
to select those who will initially enter the program, and the teach- 


used for final selection, Assuming the school and work-training pro- 
gram involved in the present, study to be representative of both the 


screening procedures in use, the instance where the tests are in use 
would seem to produce the greatest savings. Although 30 per cent 
of those who would be considered hirable would be screened out of 
the program by using both scales, this loss might be more than offset 
by the savings that would accrue from screening out all (100 per 
cent) of those who would presumably not be hired. 

; It remains to be seen, of course, just how specific to the condi- 
tions, Ss, and Program setting the present results are. Further re- 


ro may or may not show they have some more general applica- 
lity. 
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PREDICTING SUCCESS IN A VOCATIONAL 
REHABILITATION PROGRAM WITH THE RAVEN 
COLOURED PROGRESSIVE MATRICES 


KENT L. KILBURN anp ROBERT E. SANDERSON 
Porterville State Hospital 
California Department of Mental Hygiene 


Tuerm has been interest in the prediction of vocational success 
by psychological tests of intelligence for several years as evi- 
denced by the work of Shafter (1957) and Walker (1951). The use 
of other types of psychological tests in this capacity has been ex- 
plored by Arnholter (1962), Ferguson (1958), Kolstoe (1961), 
Novis, Marra, and Zadrozny (1960), Taylor (1963), and Warren 
(1961). Madison (1964) and Tobias and Gorelick (1963) have 
shown a positive relationship between intellectual level and voca- 
tional success. More specifically Appell, Williams, and Fishell 
(1962), Fry (1956), and Jackson and Butler (1963) have noted the 
utility of the Wechsler Scale in this area. However, Reger and Daw- 
son (1961) indicated that a Wechsler score did not adequately pre- 
dict occupational therapy success, and Parnicky and Kahn (1963) 
found no relationship between Peabody Picture Vocabulary scores 
and vocational success. Aside from the limited success noted for 
Wechsler scores, the prediction of vocational success with psycho- 
logical tests, particularly intellectual tests appears to have not been 
determined. 

This paper is concerned with assessing the degree to which the 
Raven Coloured Progressive Matrices (RCPM) (Raven 1956a) 
and the Peabody Picture Vocabulary Test (PPVT) (Dunn 1959) 
predict patient success in a vocational rehabilitation program (VR) 
at a hospital for the mentally retarded. When the VR program was 
reorganized in June of 1964, selection of each of the 40 patients 
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for the program was made primarily on the criteria of his pa 
activities, current behavior, and general potential based on a sub=- _ 
jective evaluation of the patient’s speech and socioadaptive behav- 
ior level. 

Independently of this selection process, all patients who were 
considered for the VR program were tested with the RCPM, PPVT, 
and the Stanford-Binet word vocabulary. The staff of the VR 
program had no access to the RCPM or PPVT test results, Thus, 
their decisions on patient disposition in the program were made ~ 
without knowledge of these data. 4 

During the 18 months that the VR program has been in opera- — 
tion, 12 patients have been placed successfully into a work situa- 
tion outside the hospital, and 9 patients have been dropped from ~ 
the program as failures because of the lack of adequate work per- 
formance or displays of asocial behavior as determined by the VR 
program staff. As these patients were moved out of the program, new 
ones were added to keep the number at approximately 40. 


Methods and Results 


The group of 21 patients were split into the 12 successes and the 
9 failures. A ¢ test for the difference between mean scores and 
point biserial correlation were computed between each sub-group 
for each test in order to determine the power of the raw scores of 
the RCPM and PPYT to predict success in the VR program. 

The results shown in Table 1 indicated a difference significant 
beyond the .05 level between the RCPM mean score of 16.46 for 
Success, and 12.56 for failure. The Point biserial correlation be- 


TABLE 1 
Distribution of Scores for VR Patients 
t for Mean 
Mean 8D N  Range Difference rp 
Success 16.46 4.99 12 11-24 
RCPM i 2.73* -60** 
Failures 12.56 2.24 9 9-16 
PPYT Success 70.58 14.83 12 53-106 
1.18 27 


Failures 64,11 10.18 9 51-83 ; 
* Significant beyond the .05 level. 


be the .01 level. 
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tween success or failure and RCPM scores of .60 was significant 
beyond the .01 level. 

However, the difference between the PPVT mean score of 70.58 
for success and 64.11 for failure was not significant. Similarly, the 
point biserial correlation between success or failure and the PPVT 
scores of .27 was not significant. 

Discussion 

The lack of a significant relationship between PPVT scores and 
success or failure in the VR program is similar to the finding of 
Parniky and Kahn (1963) who indicated that a test of verbal 
intelligence does not relate to vocational placement. The significant 
positive relationship between RCPM scores and status in a voca- 
tional training program is supported by the findings of Appell, Wil- 
liams, and Fishell (1962) who found Wechsler scores to be useful 
in predicting vocational training success or failure. The RCPM 
and Wechsler both showing positive relationships to VR success 
seems reasonable, as Orme (1961) found a positive correlation of 
.93 between the Wechsler Adult Scale and the RCPM in a sample 
of 203 retarded subjects. The RCPM described by Raven (1956b) 
as “a test of observation and clear thinking” indicates an impor- 
tant aspect of success in a VR program. Success in such a training 
program involving actual prevocational experiences seems to be 
more related to the ability to observe and to solve new problems 
than merely to recall word meanings. Although an interesting trend 
has been observed, caution should be applied in generalizing the 
present results. This trend should be re-examined with larger 
groups of various types of VR training programs. 
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THE PREDICTIVE VALIDITIES OF SELECTED APTITUDE 
AND ACHIEVEMENT MEASURES AND OF THREE PER- 
SONALITY INVENTORIES IN RELATION TO NURSING 

TRAINING CRITERIA 
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Problem 


For a sample of one hundred freshman trainees in student nurs- 
ing for the academic year of 1964-1965, the two-fold purpose of 
this investigation was (1) to obtain additional cross-validation 
data on a number of cognitive and non-cognitive predictor vari- 
ables that had been employed during previous years in the selec- 
tion of candidates for participation in a nursing training program 
at the Los Angeles County Hospital and (2) to derive new informa- 
tion concerning the predictive validities of scales in the Edwards 
Personal Preference Schedule (EPPS) and in Cattell’s Sixteen Per- 
sonality Factor (16 PF) Questionnaire relative to both academic and 
clinically oriented criteria. In addition, it was also desired to 
check out the predictive validities of the previously employed Min- 
nesota Multiphasic Personality Inventory (MMPI) scales with 
respect to certain items on the Ward Performance Seale that had 
been designed to represent operational statements of several of the 
constructs in the EPPS and in the 16 PF Questionnaire. The 
constructs selected were judged to reflect personal needs thought 
to be important by supervisory staff to success in day-to-day nurs- 
ing activities in the wards of any large metropolitan hospital. 


Predictor and Criterion Variables 


In Table 1 the ten predictor variables representing measures of 
scholastic aptitude and achievement are enumerated along with the 
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fourteen criterion measures employed. Only those noncognitive 
predictors (subscales of two personality inventories—the EPPS 
and the 16 PF Questionnaite) are cited that showed statistically 
significant (.05 level) predictive validity coefficients with three or 
more of the criterion measures. (The MMPI scales are not listed 
because none of them satisfied the statistical requirement just de- 
fined.) 


Statistical Treatment 


With the exception of the predictor represented by grades in 
high school chemistry which were placed on a four-point scale, all 
measures were converted to stanine scores by a normalized-rank 
method (Haney, Michael, and Gershon, 1962). Through use of an 
IBM 7090 program at the Western Data Processing Center at 
UCLA, product-moment coefficients of correlation were calculated 
among all possible pairings of variables. Since there was a marked 
restriction of range as evidenced by the fact that the standard de- 
viations of scores for the total group of several hundred applicants 
were from 25 to 80 per cent larger than those of the survival 
group, the coefficients reported in Table 1 are minimal in size. An 
example of such restriction is apparent from the fact that on the 
total scores of the California Reading Test and on the total scores 
of the California Mathematics Test trainees were customarily re- 
quired to place at least above the 50th centile with respect to pub- 
lished 12th grade norms, 


Results 


The following major findings may be summarized for the data in 
Table 1: 

1. For the academic criteria the part and total scores of the 
California Reading Test, and the Reasoning and total scores of the 
California Mathematics Test showed for the most part statistically 
Significant coefficients of predictive validity. However, neither of 
these instruments appeared to be predictive of rated characteristics 
of performance in the wards. 

2. With the exception of the criterion variable of grades in Psy- 
chology II, the somewhat speeded measure of spatial ability— 
EAST No. 5—yielded one positive and twelve negative coefficients 
which failed to attain statistical significance. 
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3. As in previous studies (e.g. Haney, Michael, and Gershon, 
1962; Michael, Haney and Gershon, 1963; Michael, Haney, and 
Brown, 1965) high school grade point average (GPA) whether 
in all subjects or in academic subjects alone served as the best over- 
all predictor of success in the academic portion of the training pro- 
gram. In the current sample, the GPA in high school chemistry did 
not afford a single predictive validity coefficient that was not sur- 
passed by the high school GPA in all courses. Moreover, none of the 
GPA variables was even moderately predictive of success in ward 
performance. 

4. Although failing to be predictive of success in ward perform- 
ance, two scales in the EPPS—order and autonomy—and two 
scales in the 16 PF Questionnaire—L and O—did show statistically 
significant degrees of relationship with at least three academic cri- 
terion measures. It would appear from a study of the size and sign 
of coefficients for the two EPPS scales that relatively high stand- 
ings in the trait order and relatively low placement in the trait 
autonomy were slightly predictive of success in the academic pro- 
gram. Designating a trustful (adaptable) versus a suspecting (jeal- 
ous) and a confident (unshakable) versus an insecure (anxious) 
orientation, the L and O scales, respectively, in the 16 PF Ques- 
tionnaire yielded for this sample validity coefficients which sug- 
gested that for at least three of the academic criteria trainees who 
were trustful and confident were more likely to succeed in the 
academic program than were those who were suspicious and insecure 
in their adjustment to academic requirements. 

5. As in previous studies with nursing trainees at the Los Angeles 
County Hospital the degree of correlation among grades in academic 
courses was usually higher than that found between cognitive pre- 
dictors and grades in these same courses. As was pointed out else- 
where (Michael, Haney, and Gershon, 1963), the presence of a mod- 
erate degree of correlation among course marks (usually between 
bes and fa Suggested one or more of the following hypotheses: (a) 
a “halo” effect in grading Practices, (b) a motivational syndrome 
related to earning grades, (c) an academic savoir-faire factor em- 
Way's sr ik of students to the needs and expectations of 

eir teachers, and (d) an institutional press in which students 


respond to imposed standards of behavior in a generally conform- 
ing manner. 
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6. Although the presence of moderate coefficients of correlation 
(which are not reported here, but customarily were found between 
30 and .70) among several of the pairs of the twelve items of the 
Ward Performance Scale at both the May and June evaluations 
might indicate that the characteristics were intercorrelated to some 
degree, the likelihood of a general halo effect in the rating process 
could not be discounted despite the care with which instructions on 
rating procedures had been communicated to the supervisory staff. 
The median test-retest coefficient of correlation of each of the twelve 
characteristics evaluated in May with each of the eleven other 
characteristics and with itself evaluated in June was .37. For the 
May evaluation and for the June evaluation the median coefficients 
of correlation of every one of the twelve characteristics with each of 
the other characteristics (but not with itself) were .54 and .48, re- 
spectively. (Application of the Spearman-Brown formula yielded 
internal consistency estimates in reliability of .94 and .92 for the 
May and June evaluations.) 

7. Despite the fact that each of the twelve characteristics in the 
Ward Performance Scale was developed to measure activities that 
Were judged coordinate with constructs in the EPPS and to some 
extent with those in the 16 PF Questionnaire the coefficients of cor- 
relation observed between measures of the constructs in the inven- 
tories and the corresponding characteristics evaluated in the Ward 
Performance Scale hovered near zero and approached significant 
values at only a chance level (i.e., for about five per cent of the 
entries obtained). Thus neither the scholastic aptitude and 
achievement measures nor the scales in the MMPI, in the EPPS, or 
in the 16 PF Questionnaire were predictive of the measures of per- 
formance in the ward. It should also be noted that even the median 
Coefficients of correlation of each of the twelve characteristics on 
the Ward Performance Scale with each of the other criterion meas- 
ures in the academic program approximated values that were either 
barely significant at the .05 level or only close to significance. 


Conclusion 


For a sample of one hundred freshmen in student nursing who 
Survived one year of training at the Los Angeles County Hospital 
moderate and statistically significant coefficients of validity for 
Measures of high school GPA, reading comprehension, and mathe- 
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matics reasoning were found relative to academic courses in the 
program. For the most part scales of the MMPI, EPPS, and the 16 
PF Questionnaire showed a lack of validity either for academic 
course work in nursing training or for rated performance in ward 
activities. The presence of moderate to high intercorrelations among 
marks in the academic courses as well as among items on the Ward 
Performance Scale suggested not only the probable presence of a 
halo effect but also a motivational syndrome for earning high 
marks in both academic and clinical activities. 
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A PLACEMENT STUDY IN ANALYTIC GEOMETRY AND 
CALCULUS! 


RICHARD L. FRANCIS 
Southeast Missouri State College 


Ir is the purpose of this article to present certain portions of a 
recent placement study in mathematics at the University of Mis- 
souri. In particular, an effort was made to assess the values of se- 
lected test scores for the prediction of freshman success in the 
first and second courses of a three semester sequence in analytic 
geometry and calculus. Since the study was concerned with fresh- 
men, the third course was not considered. These two integrated 
courses in analytic geometry and calculus are ordinarily referred to 
as Calculus I and Calculus II. 

As to pre-requisites, a grade of C or better was required in Trig- 
onometry and College Algebra in order to qualify for Calculus I. 
Otherwise one had to be a select student with 3.5 to 4 high school 
units of mathematics including two units of algebra, one unit of 
geometry, and one-half unit of trigonometry. As calculus seemingly 
demands proficiency in algebraic and trigonometric concepts and 
manipulation, it was necessary that any uncertainty here as to 
qualifications be resolved. Also the high number of deficient grades 
received in Calculus I and Calculus II gave evidence of the need for 
a valid basis for assigning pupils to these courses. 


Method of Analysis 


Scores on the Mathematics Advanced Placement Tests (Algebra 
and Trigonometry) were taken as two of the variables in the regres- 
sion analysis for the prediction of Calculus I achievement. These 
c . 

1 The writer wishes to thank Dr. H. W. Schooling and Dr. F. G. Delon, his 
advisors in this study, for their most valuable help and suggestions. 
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tests were prepared by the Missouri Section of the Mathemat- 
ical Association of America and are given annually at the uni- 
versity for the purpose of advanced placement. This is a three hour 
examination with 130 points possible on the algebra section and 70 
points possible on the trigonometry section. One additional predic- 
tor variable was used: the School and College Ability Test quanti- 
tative score. This test is administered to all freshmen having less 
than 24 semester hours of credit; thus these scores were available 
for the entire experimental group. There were 50 points possible on 
the quantitative examination. 

The construction of the test instrument to measure achievement 
in Calculus I was one of the initial steps in the experiment. In 
order to ensure comprehensiveness, a survey was conducted in- 
volving the teachers of calculus in the department of mathematics. 
Also, test papers of the instructors were examined to determine the 
frequency and types of errors. The information obtained was used 
as a basis for determining the content of the achievement examina- 
tion. 

For the final form of the achievement test, the thirty most valid 
items were selected from the sixty items initially constructed. The 
discriminating power of each item was determined by administer- 
ing the first form of the examination to a select group of students. 
By considering the proportion of correct responses in the upper 
and lower subgroups, the validity index for each item was read 
directly from a table of normalized biserial coefficients. The method 
of rational equivalence (Richardson and Kuder, 1939) was used to 
Provide a measure of reliability. This coefficient was obtained by 
using the scores of the forty people in the pilot group and was found 
to be .769 with the standard deviation equal to 4.892. To further 
substantiate the reliability of the examination, a coefficient was 
calculated based on the 569 students taking the Calculus I Achieve- 
ment Examination in January of 1965. Here the reliability coeffi- 


A was found to be .778 with the standard deviation equal to 


Findings of the Calculus I Study 


Scores on the Mathematics Advanced Placement Tests were avail- 
able for only 254 students of the freshman group enrolling in Cal- 
culus I. SCAT-Q scores were available for all of these however. 
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By least squares analysis using the score weights for Algebra (X2), 
Trigonometry (X), and Quantitative (X4) the various regression 
equations of Table 1 resulted for the prediction of Calculus I achieve- 
ment (Xı) scores. The coefficient of multiple correlation between 
the scores obtained on the criterion variable of Calculus I achieve- 
ment (X,) and the combined action of the independent variables 
Xo, Xs, and X4 was .479. The SE est, x1) Was equal to 4.073. 

The best single predictor of Calculus I achievement (X,) was 
the Algebra score (X2). The correlation between the criterion and 
this measure of mathematical ability was .402 (significant at the .01 
level). The various intercorrelations are listed in Table 2. 

The point-biserial coefficient of correlation was calculated for 
the pass-fail dichotomies of teacher assigned marks in Calculus 
I and learning achievement in Calculus I as measured by the Cal- 
culus I Achievement Examination. The value 1 was assigned to a 
passing grade (C or better) in the course and the value 0 was 
assigned to a failing grade, with the test criterion being the score 
on the achievement examination. This correlation was found to be 
526 which is significant at the .01 level. The calculation of this 
Coefficient seemed important, since the study was limited by the 
teacher variable as it pertained to assigned marks in Calculus I and 
Calculus II. 


Findings of the Calculus II Study 


The size of this second experimental group was 143 and consisted 
of those from the Calculus I study who enrolled in Calculus II. 


TABLE 1 
Regression Equations and Standard Errors of Estimate of Calculus I 
Achievement Scores for a Combination of the Variables Algebra, 
Trigonometry, and Quantitative (N = 254) 


Variables Combined Regression Equations BE (ost. x.) 
Algebra (X,) and 
in prety (Xs) Ê, = 16.4906 + .07996Xs + -03288X: 24,236 
ra (Xs) and 
Quantitative (X) Ê, = 4.57174 + .06415X2 + -29764X4 +4.097 
gonometry (X+) Tror 
ee Quantitative (X,) £, = 1.19767 + .08082Xs + -4119X 24.19 
ans 2), Trigonom- 
Ba 4.073 


Quantitative (x4) Ê, = 3.73188 + .05482X2 + .04274Xs + 30767 X. 
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TABLE 2 


Intercorrelations between a Measure of Achievement in Calculus I 
and Selected Measures of Mathematical Ability (N = 254) 


Coefficients of Correlation 


Test Xs Xs X, Xı 


a EE ic 
Algebra (X+) — -3880 -3929 4015 
Trigonometry (X;3) — — .0789 .2228 
Quantitative (X4) — — — .3814 
Calculus I (X1) — at a a 


Og o O o U O S 


Regression equations involved the numerical equivalent of the se- 
mester grade received in the course, called the performance or crite- 
rion, and was denoted by Y. The letter grades A, B, C, D, and F 
were assigned the respective values 4, 3, 2, 1, and 0. The Calculus 
I achievement, algebra, trigonometry, and quantitative vari- 
ables from the previous study became the predictor variables. The 
various regression equations are listed in Table 3. The coefficient of 
multiple correlation between the scores obtained on the criterion 
variable of Calculus II achievement (Y) and the combined action 
of the independent variables Xı, Xə, Xs, and X, was .607. The 
SE est, x) Was equal to .901, 

The best single predictor of Calculus IT achievement was the Cal- 
culus I (X1) score. Here r was equal to .521 (significant at the 01 
level). The various intercorrelations are listed as Table 4. 


Since more than half of the variability of achievement in Cal- 
culus I and Calculus II was not explained by the measures con- 
investigation is suggested. Answers 


ons could perhaps result from this further 


sidered in this study, further 
to the following questi 
investigation: 

1. What specific study habits are associated with a high level 


of achievement in Calculus I and Calculus II? 
2. Is the effect of such 


stability in the leami 
urable? 
eens findings of this study indicate a need for more valid 
placement instruments. What procedures are most effective 


qualities as interest, motivation, and 
g of calculus quantitative and meas- 


Concluding Remarks 
x 
| 
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TABLE 3 


Regression Equations and Standard Errors of Estimate of Calculus II Achievement 
Scores for a Combination of the Variables Calculus I Achievement, Algebra, 
Trigonometry, and Quantitative (N = 148) 


Variables Combined Regression Equations SEvest. y) 
Calculus I (X1) and 

and Algebra (X2) = —1.54713 + .12990X1 + .01589X_ =. 90063 
Calculus I (X1) and 

Trigonometry (X 3) = —1.73085 + .16100X, + .01772X3 +.94012 
Calculus I (Xi) and 

Quantitative (X4) = —2.66591 + .16438X1 + .02692X, +.95705 
Algebra (X+) and 

Trigonometry (X4) f= 89947 + .02137X: + .01135Xs +.97996 
Algebra (X2) and 

Quantitative (X4) Y= 35757 + .02249X. + .01580X, + .98624 

? Trigonometry (X+) and 
Quantitative (X4) = —1,22866 + .02558X, + .06648X 4 1.05551 


Calculus I (X1), 

Algebra (X2), and 

Trigonometry (Xs) LY = —1.61559 + .12809X, + .01430X: + .00927X,  +.89823 
Calculus I (X1), 

Algebra (X+), and 

Quantitative (X4) f = —1.41362 + .13048X1 + .01606X. — .00342X, = .90379 
Calculus I (X1), 

Trigonometry (X3), 

and Quantitative (X4) Y = —2.78128 + .15278X1 + .01772Xs + .02689X, +.93882 
Algebra (X2), Trigo- 

nometry (X4), and 

Quantitative (X4) f= 06238 + .02008X: + .01194Xs + .01959X, .98130 
Calculus I (X,), Algebra 

Ca i petits 

a), and Quantita- 
tive (X,) 3 f = —1.61053 + .12811X1 + .01431X: + .00927X; = .90148 


f, —.00012X, 
ee EE 


TABLE 4 


Intercorrelations between a Measure of Learning Achievement in Calculus II 
and Selected Measures of Mathematical Ability and Achievement (N = 148) 


Coefficients of Correlations 
Test Ke X: Xs X, ¥ 
Calculus I (X4) 3954 .1913 - 2863 5206 
f 4807 
` Algebra (X+) -3620 ped call 


‘Trigonometry (X3) 
Quantitative (Xa) SAk 


Gee) O 


ey -2286 
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for the construction and evaluation of other freshman place- 
ment tests in mathematics? 

4, What is the relative value of high school analytic geometry 
and calculus in the determination of placement and how do 
these courses relate to achievement in college calculus? 

5. What is the value of the Mathematics Advanced Placement 
Tests in the prediction of achievement in mathematics courses 
other than Calculus I and Calculus Il? 

Because of the complexities involved in the learning of calculus, 
many factors should be considered when placing a student. Certain 
of the variables mentioned above do not lend themselves to easy 
measurement. Hence, if they are to be considered, it must be in a 
subjective way by the teacher and the advisor. The use of mathe- 
matical ability and achievement tests in the various manners 
described in this study to supplement the judgment of teachers 
and advisors should contribute to the more accurate placement of 
students in Calculus I and Calculus II. 
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THE RELIABILITY AND CORRELATES OF AN 
ACHIEVEMENT INDEX 


ERICH P. PRIEN 
The University of Akron 
AND 
DAVID E. BOTWIN 
University of Pittsburgh 


Severna methods are currently used to quantify an achieve- 
ment index (AI). The AI is defined here as the difference between 
predicted performance and actual performance. Actual performance 
may be predicted by a single measure of ability, or by using a com- 
bination of aptitude and ability measures. Graphic calculation of 
deviations from a regression line (or computations) yields a con- 
tinuous score distribution, just as does a difference between 
standard scores. Various other techniques have been used, all of 
which are intended to identify relative achievement criterion 
groups. 

The appropriate method of analysis, to study over-achievement 
and under-achievement according to Thorndike (1963), is a partial 
correlation coefficient without the computation of a separate AI. 
However, deviations were calculated in this study to permit compu- 
tation of the reliability or stability of relative achievement over 
time. The AI was then used for correlations with personality test 
Scores rather than the part correlation technique. Ae 

The primary purpose of this study was to investigate the reliabil- 
ity of an AI over a two year period. The secondary purpose was to 
determine the correlation of the AI with selected personality char- 
acteristics. 
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Method 


Subjects were the complete junior class of 101 students in a small, 
private women’s college. Complete data were obtained from ninety- 
five students, 


Measures 


The criterion data available for this study were the grade point 
averages for the freshman year (GPA-1), the sophomore year 
(GPA-2), the fall term of the junior year (GPA-3), and the cumu- 
lative performance for the first two years (GPA 1-2). 

All students had initiated their college career at the sponsor- 
ing institution and had taken a battery of aptitude and achieve- 
ment tests. A selection process based primarily on high school 
graduating class rank and to some extent on the Verbal (V) and 
Mathematics (M) portions of the Scholastic Aptitude Test (SAT) of 
the College Entrance Examination Board was in effect at the time of 
admission. However, some latitude was allowed the admissions 
authority to accommodate individual cases where past circum- 
stances indicated a reasonable probability of success. Thus, while 
some restriction in range is probable for predictor scores, the ef- 
fect is assumed to be minimal, Selective attrition, however, is more 
likely relevant to the problem of restriction of Tange. 

The personality measures used in this study were the Gordon Per- 
sonal Profile, (Gordon, 1963b), Gordon Personal Inventory (Gor- 
don, 1963a), and the Survey of Interpersonal Values, (Gordon, 
1961). These specific tests were selected by the authors because of 
Personal experience with the Measures in research and in extensive 
student counseling, Personality tests were administered to the stu- 
dents early in the Fall of their junior year, Administration was on a 
Group basis, with assurance given to the students by the college 
administration that their performance on this test would not be 
used in any administrative action as far as their group was con- 
cerned. Apparent complete Cooperation was obtained from the group. 

The AI was computed Separately for each year and for the two 
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TABLE 1 


Product-Moment Correlations, Means and Standard Deviations of Academic 
Aptitude Measures and Criterion Measures (N = 95) 


Measures 1 2 3 4 5 6 7 8 Mean Sigma 

1. SAT Verbal 45 75 73 48 41 35 46 473 89.0 
2. SAT Math 51 52 41.44 32 45 470. 76.4" 
3. OSUPE 64 60 44 37 51 93.8 22.1 
4. COOP Reading 53 45 30 53 60.8 8.2 
5. GPA-1 76 61 92 2.51 „54 
6. GPA-2 65 92 2.62 50 
7. GPA-3 65 2.67 65 
8. GPA-1, 2 (Combined) 2.57 48 

Rs.3,1—.52 ¥ 

Ra.s,2=.56 

Ro.s,4—.57 


$$ 
Note.—Decimal points omitted from correlation coefficients only. E 
* All correlations are significant at the .01 level. 


bility of the criteria. The multiple correlations (R) computed to 
predict GPA 1, 2 when the OSUPE and SAT verbal were used was: 
.52; when the OSUPE and SAT mathematics were employed, R was 
.56; when OSUPE and COOP reading C-2 were chosen, R was 57. 
The question of using a multiple regression equation to calculate 
the AI was resolved after consideration of the point in time of 
administration of each test. The SAT scores were obtained some 
time prior to entrance, the OSUPE at the beginning of the fresh- 
man year, and the COOP Reading C-2 at various times during the 
freshman year or at the beginning of the sophomore year. Since 
there is already a question of temporal separation to be considered 
in the interpretation, further confounding seemed undesirable. The 
intercorrelation of aptitude test scores with the individual year and 
cumulative GPA is shown in Table 1. 

In this study, relative achievement was a continuum, and the 
entire range of the variable was used instead of employing extreme 
groups as had Keimowitz and Ansbacher (1960), Krug (1959), and 
Diener (1960). Computation of the AI was done graphically from 
the plotted regression line. Accuracy was judged by the authors as 
approximately -+.02, of a grade point. Thus, if the predicted GPA 
was 2.55 and the actual GPA was 2.89, the AI would be -34 + .02. 
A small sample of calculations was made and compared with the 
much faster graphic approach and the accuracy was deemed satis- 
factory for present purposes. Negative values were eliminated from 
the AI by the addition of a constant. 
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Results 


The significant correlations between the AT computed separately 
for the first year, second year, and first term of the third year, indi- 
cated the existence of true variation in the GPA which is independ- 
ent of academic aptitude. These data appear in Table 2. 

Product-moment correlation coefficients which were computed be- 
tween personality measures and GPA 1, 2 and with the AI LA 


TABLE 2 


Intercorrelations and Standard Deviations of Individual Achievement I; ndices 
Calculated for the Freshman, Sophomore, and First Term 


of the Junior Year, (N = 95) 
Year 2 3 i 
3 -59 
Note. —A1 correlations are significant at the .01 level. 


icant at the .05 level, 


TABLE 3 


Correlations, Means, and Standard Deviations of GPA 1, 2, and 
Al, and Personality Variables (N = 95) 


Variable OSUPE GPA(1,2) AI(1,2) Mean Sigma 
Gordon Personal Profile 


Ascendancy 05 —05 —07 3.59 8.4 
sponsibility —04 17 28** 3.73 6.1 
Emotional Stability —10 06 21* 2.89 6.7 
Sociability 01 ~19 ~19 4.55 6.8 
Gordon Personal Inventory 
Cautiousness —06 27** 43** 1.58 6.4 
Original Thought 27** 36** 25* 6.00 6.4 
Personal Relationg —20* 08 24s 4.06 6.4 
Vigor 03 17 14 4.21 5.8 
een of Personal Values 
lupport —02 
Conformity Iae 9, he foes 57 
gnition ~01 ~03 00 10.57 4.3 
Independence 04 10 05 20.49 6.2 
Benevolence mld | oy 13 16.47 63 
lership 1 06 —05 12.07 6.2 
Mean 93.78 2.57 00 


Sigma 22.08 48 39 
Note.—Deci; I poi: p x k 
$ Signife 


it at the .05 level, | 
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appear in Table 3. As is readily apparent, there is considerable simi- 
larity of correlations with the GPA 1, 2 and the AI 1, 2. Correla- 
tions which appear between the personality variables with GPA 1, 
2 and with the AI 1, 2 cannot be construed as representing causal 
relationships. Rather, it can only be said that the significant cluster 
of traits, in either case, describes students who tend to receive 
relatively higher or lower grades than do their peers. 

With the contribution of academic aptitude eliminated, relative 
achievement is significantly related to personality characteristics. 
The cluster of variables shows positive correlations with cautious- 
ness, responsibility, personal relations, emotional stability, and 
negative correlation with sociability. The cluster suggests that 
relatively high achieving students behave differently or at least 
says that they do behave differently than do low relative achievers. 
We may speculate that the high achievers are actually more pro- 
ductive, more accurate, more painstaking, and thus, better students 
who deserve relatively higher grades, rather than beneficiaries of 
rater halo. 

Generally, the results obtained in this study are similar to those 
found by Krug (1959) who described the overachiever as scoring 
higher than underachievers on the Edwards Personal Preference 
Schedule (Edwards, 1959) subscales Achievement, Order, Endur- 
ance; and lower on the subscales of Affiliation and Heterosexuality. 
The behavioral description offered by Krug corresponds to that es- 
poused in this study. i 

Finally, whether one subscribes to the interpretation of instruc- 
tor halo, or of the actual fulfillment of student role behaviors result- 
ing in real superiority of performance, the relationship between 
personality factors and relative achievement is significant. 
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PERSONALITY AND GRADES OF COLLEGE STUDENTS OF 
DIFFERENT CLASS RANKS! 


RICHARD M. SUINN 
University of Hawaii 


Srupims of the relationship between personality and college grades 
have used various types of questionnaires and projective tests. Of 
such, the Guilford-Zimmerman Temperament Survey (Guilford and 
Zimmerman, 1949) should be of greatest value since it measures 
traits more appropriate for describing the normal college student, 
The purpose of this study was to examine the relationship be- 
tween Guilford-Zimmerman scores and grades on upper and lower 
division students. 


Method 


Results from the Guilford-Zimmerman test were obtained on 184 
private liberal arts college students: 47 freshmen, 46 sophomores, 
46 juniors, and 45 seniors. Grade-point averages for the last semes- 
ter were obtained. For the freshmen, grade predictions were also 
available. These latter were based on weighted Verbal and Math 
scores of the Scholastic Aptitude Test of the College Entrance 
Examination Board (CEEB) and high school grade-point average 
from previous studies of incoming freshmen. 


Results 


Grade-point average was significantly correlated with seriou 
for freshmen (r = .43, p < .01), juniors (r= 4, p < 01), 
all students combined (r = 27, p < -01); and with friendliness for 
sophomores (r = .32, p < .05), seniors (r = 33, p < .05), and all 
students combined (r = 21, p < .01). 
dyve supported in part from a grant from the National Institute 
of Mental Health, grant no. M.R. 07490-01. 
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For the freshmen men and women, grade prediction scores were 
subtracted from their actual grade achievements to give deviation 
scores for use in correlations. Since the grade prediction formula 
included CEEB scores, this method controlled for differences in 
scholastic aptitude. Results showed that seriousness and the degree 
to which a student performed better than predicted were signifi- 
cantly related (r = .44 for men; .43 for women, p < .05). Thus, 
students described as serious or as showing restraint tended to 
achieve grades higher than had been expected of them from their 
high school record and college aptitude scores, 
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OTIS PREDICTION OF GRADUATE EDUCATION 
COURSE GRADES 


A. M. FOX ax L. L. AINSWORTH 
Sam Houston State College 


Tux purpose of this study was to determine the extent to which 
Otis Quick-Scoring Mental Abilities Test (0QS) 1Q’s may be useful 
for the prediction of grade-point ratios (GPR’s) in graduate Edu- 
cation courses. 


Sample 


The sample consisted of 306 students at Sam Houston State 
College for whom undergraduate OQS 1Q’s and at least one grade 
in a graduate Education course were available, for the period July, 
1959, through August, 1964. 


Procedure 


GPR’s were calculated, and then relationships of OQS 1Q’s to 
GPR’s were determined by the Stepwise Regression Analysis Pro- 
gram (STRAP) in an IBM 1620 Data Processing System. An F- 
Tatio of 4.0 was used as the criterion in testing the significance of the 
relationships. Significant relationships were found for OQS IQ’s 
with total GPR (GPRT), basic required courses (GPRB), Meas- 
urement and Evaluation (GPR93), Secondary Curriculum 
(GPR94), Human Growth and Development (GPR97), superin- 
tendent’s certificate courses (GPRS), supervisor’s certificate courses 
(GPRY), and principal’s certificate courses (GPRP). Regression 
equations, coefficients of alienation (k), and indices of forecasting 
efficiency (Æ) were then computed for these significant relation- 
ships. 
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Results 
The results of this study are summarized in Table 1. 


TABLE 1 
Relationships of OQS to Dependent Variables 


OQS IQ GPR 


Regression 


k Mean S.D. Mean S.D. Equation 
0.33 0.94 111.6 9.7 3.2 2.2 + 0.018 (IQ) 
0.32 0.94 111.7 96 3.1 1.9 + 0.020 (IQ) 
0.31 0.95 112.1 9.7 3.2 1.8 + 0.021 (IQ) 
0.26 0.97 112.4 9.2 3.2 2.1 + 0.018 (IQ) 
0.37 0.93 112.4 9.0 3.0 0.8 + 0.029 (IQ) 
0.37 0.93 111.5 8.7 3.3 1.1 +0.029 (IQ) 
0.32 0.94 110.4 8.6 3.2 2.2 + 0.018 (IQ) 
0.35 0.94 110.7 88 3.1 1.9 +0.020(1Q) 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 
1966, 26, 1057-1062. 


CONCURRENT VALIDITY OF THE GATES LEVEL 
OF COMPREHENSION TEST AND THE BOND, 
CLYMER, HOYT READING DIAGNOSTIC TESTS 


DONALD A. BENZ 
Wisconsin State University 4 
at Stevens Point ` 
AND 
ROBERT A. ROSEMIER 
Northern Illinois University? 


Tum purpose of this study was to investigate the relationship 
between word-analysis skill proficiency to reading comprehension 
performance among fourth graders. This attempt differs from 
many prior studies in that it did not structure the sample according 
to intelligence or other factors and thereby restrict this investiga- 
tion of the relationship to “over” or “under-achievers.” Rather, it 
allowed a determination among these variables on children “as they 
were.” It should contribute additional information concerning the 
“substrata model” of the reading task as described by Holmes and 


Singer (1964). 
Variables 

The six word-analysis skills selected for investigation were: Syl- 
labication, location of root words, words in context, word eee 
beginning sounds, and rhyming sounds, as defined by the Bom, 
Clymer, Hoyt (1955) Silent Reading Diagnostic Tests, Form D-A. 
Reading comprehension, serving as the criterion variable, was as- 
sessed by the Gates (1958) Level of Comprehension Test, Type 
LC, Form 3. 


1 Presently with the Southwest Regional Laboratory for Educational 
search and Development, Los Angeles. 
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Subjects 


The participants consisted of all fourth grade children attending 
public schools in six towns and cities in five northeastern states. 
These fifty-five classrooms of 1490 children included what the 
authors believed to be a representative cross-section of school en- 
vironments; included were a college town (N =273), a resort 
village (N = 148), a state capital (V = 115), an industrial-pro- 
fessional community (N = 274), an agricultural center (V = 
248), and a so-called “bedroom” community serving a nearby 
metropolitan center (V = 432). Although none of the communi- 
ties was considered “culturally deprived,” a wide range of environ- 
ments did exist among and within the systems considered. The 
number of classrooms in the systems ranged from five to eighteen. 
The population appeared to be only slightly above normal (mean = 
107) on the basis of intelligence scores which were available for 
94 per cent of the group from a variety of tests, 


Procedures 


Classroom teachers were trained by the investigators to admin- 
ister the tests which were given in consecutive half-day testing 
sessions during a one month interval. The tests were scored by the 
investigators. Of the 1490 fourth-grade children tested, those scor- 
ing between 6.0 and 92 on the Gates test were considered high 
level readers (V = 474), those between 4.6 and 5.9 as middle 
level (V = 450), and those from 2.0 to 4.5 as low level (V = 
478). A number of subjects were dropped from future analyses 
either because of absence during testing or because of failure to 


achieve the minimum level reported on the comprehension test. As 
& result, 1402 children were available for analysis, 


Analyses 
; The analyses which were performed Tesulted from the following 
lines of reasoning. If a skill was contribu 
hension, it should discriminate among various proficiency levels of 
reading comprehension, Further, it should show a high degree of 


: s moved or “partialed out.” When 
taken as a whole, these six word analysis skills should account for 
much of the existing error variance in reading comprehension. 


— 


dram 


n 


BENZ AND ROSEMIER 1059 


The first approach required an analysis of variance and sub- 
sequent Scheffé tests (Hays, 1963, pp. 483-485) among three levels 
of student reading comprehension for each of the six word-analysis 
skills. The second involved the computation of a partial correlation 
coefficient between reading comprehension scores and scores on & 
given skill, while the effects of the other five skills were held con- 
stant (or removed from the correlation), for each of the six skills 
(Ezekiel and Fox, 1959, pp. 192-197). The third interest involved 
a step-wise multiple regression analysis (Smillie, 1962). 


Results 


The results of the first phase indicated that each of the six word- 
analysis skills did discriminate among the various levels of reading 
proficiency. The mean scores and the standard deviations on each 
skill for each of the three reading comprehension levels are shown 
in Table 1. 


TABLE 1 


Mean-scores and Standard Deviations Attained by High, Middle, and Low 
Reading Comprehension Levels for Siz Word-Analysis Skills 


High Middle low 
Skill & 8 & 8 2 g 
Words in context (1) 25.46 2.015 23.22 2.640 20.58 3.521 
Syllabication (2) 15.98 3.503 13.15 3.517 10.70 Ad 
Root Word (3) 28.85, 3.730 22.07 3.904 19.09 4 
Word Elements (4) 26,69 2.428 24.93 2.016 22.59 snie 
preane Sounda (0) i MAAE 22.86 1338 Tou 4672 

e won } N=450 N=418 
ZS O ü OOO 


The summary table for the simple randomized analyses of vari- 


ance are presented in Table 2. ; th 
These analyses and the ensuing tests of differences between the 


various pairs of levels, which were carried out according to pro- 
cedures developed by Scheffé and deseribed in Hays (1963, pp. 
483-485), indicated the higher comprehension readers performed 
better than middle comprehension readers who performed better 
than low comprehension readers on each word analysis skill at the 


01 level of significance. 


tween reading comprehension scores and scores on & given word 
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analysis skill while the effects of the remaining five word analysis 
skills were statistically removed. These coefficients are shown in 
Table 3. i ; 
The standard-score regression weights for the six word-analysis ` 
skills resultant in the last phase are also shown in Table 3, as are 
the standard errors and ¢-values for testing their significance in 
the regression analysis. The intercorrelation matrix among the six 
word-analysis skills and reading comprehension employed in this 
multiple regression analysis is shown in Table 4. When these word- 


TABLE 2 
Summary Table for Analysis of Variance among Levels (L) by Skills 
Source df ms F 
Skill Between L’s 2 2,849.392 364.70 R 
1 Within L’s 1,399 7.813 
Total 1,401 
Skill Between L’s 2 3 321.956 277.34 ni 
2 Within L’s 1,399 11.978 
3 Total 1,401 
Skill Between L’s 2 2,747 .657 164.75 bd 
3 Within L’s 1,399 16.678 
0 Total 1,401 
Skill Between L’s 2 2,012.561 207.47 , 
4 Within L’s 1,399 9.701 
_. Total 1,401 
Skill Between L’s 2 1,629.822 126.10 4 
5 Within L’s 1,399 12.925 
„_ Total 1,401 
Skill Between L’s 2 5,032.852 292.01 ? 
6 Within L’s 1,399 17.235 
Total 1,401 


* Significant at .01 level. (P = 4.61, df = 2,1399). 


TABLE 3 


Partial Correlation Coefficients, Beta (8) Wei 
n eights, Standard Errors 
of Beta-weights, and t-values for Testing Significance 


of Six Word-Analysis Skills 
Skill Beta-weight os t Partial r’s 
Words in context 318903 
sin ¢ i 0168 18. 99** -5548 
Syllabication -182966 .0144 12.78 -4300 
Roit Word -110184 «0124 8.89** -3230 
ord Elements 067276 0095 7.08** -2651 
Seton g Sounds -010058 -0041 2.47* -0975 
yming Sounds 229511 0154 — 14.95** 4193 


* Significant at 05 level @ = 1, = 
** Significant at .01 level (¢ aie a 
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TABLE 4 


Intercorrelation of Skill Scores and Reading Comprehension 
Scores (N = 1,402) 


Test w% @ @® 4 © © 
Words in Context (1) 544 497 .517 .438 .565 .642 
Syllabication (2) (480.641 .449 .531 .572 
Root Words (3) .433 .337 .425 .486 
Word Elements (4) .596 .573 .516 
Beginning Sounds (5) „551 435 
Rhyming Sounds (6) .598 


Reading Comprehension (7) 


analysis skills were considered in the analysis, the multiple cor- 
relation coefficient was .734—a finding which indicated that com- 
posite, optimally-weighted tests accounted for approximately 54 per 
cent of the variation in reading comprehension. 

A subsidiary check was performed on these last two analyses by 
also considering the variable of intelligence. It should be noted that 
although intelligence scores were available for 94 per cent of the 
sample, these scores were attained from a variety of intelligence 
tests. Hence any interpretation which is made from the addition of 
this variable is, at best, a rough estimate of its contribution. 

The partial correlations of each of the six word analysis skills 
and reading comprehension mentioned earlier differed by no more 
than .02 when intelligence was added. The multiple regression co- 
efficient was raised only to .753 and the percentage of variation ac- 
counted for by the six measures of skills with the addition of in- 
telligence variable was increased to approximately 57 per cent. 


Conclusions 
word analysis skills seemed to be signifi- 
cantly related to reading comprehension performance of fourth 
graders on the basis of the discrimination among various levels of 
ch skill. Five of the six skills (omit- 
ting the skill of “beginning sounds”) appeared to contribute sig- 
nificantly on the basis of their partial correlation coefficients, as 
was also observed from tests of significance of the beta weights 
within the regression analysis. Approximately 46 per cent of the 
variance in the measure of reading comprehension was not ac- 


Proficiency on the six 


reading comprehension on ea! 
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counted for by the six word-analysis skills and must be considered 
attributable to extrancous variables not under consideration. 
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STABILITY OF MMPI SCALES OVER FIVE 
TESTINGS WITHIN A ONE-MONTH PERIOD? ? 


JEROME D. PAUKER 


Department of Psychiatry 
University of Missouri Medical School 


Tue Minnesota Multiphasic Personality Inventory (MMPI) 
(Hathaway and McKinley, 1951) is frequently used in before-and- 
after studies as a measure of change resulting from the manipula- 
tion of some independent variable (as, for example, in studies of 
drug effects). The MMPI changes to be expected concommitant 
with the passage of time (without systematic insertion of interven- 
ing events) have been demonstrated in the various control groups 
used in experimental studies and in the results of retest reliability 
studies (Dahlstrom and Welsh, 1960). 

This paper presents the MMPI data of ten young women, each of 
whom took the MMPI five times during one month as part of a 
study of the menstrual cycle. The purpose of this investigation is to 
show how each of the “validity” and “clinical” MMPI scales 
(plus the Welsh A and R scales) fares in the course of such re- 
peated testing. 


Procedure 


Ten women, college students, volunteered to be subjects in a 
study of the psychological course of the menstrual cycle. They were 
required to be between 18 and 24 years of age, to have regular men- 
strual cycles of between 27 and 29 days duration, to be unmarried, 
and not to be taking oral contraceptives or other drugs which 
might affect the menstrual cycle. Seven of the ten subjects were stu- 


from the Medical and Cancer Re- 


i ta Graduate School. 
d of Dr. Auke Tellegen are 
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dent nurses in a class taught by the writer. The seven were all in 
the first year of nursing school and in the second year of college, 
The other three subjects were recruited from a sorority to which one 
of the student nurses belonged. The mean age was 20.2 years, with 
a range of from 19 to 21 years. 

All subjects had taken the MMPI on entering the university, 
and the seven nursing students had taken it again about two 
months before the start of the present testing as part of another 
study. 

Information was obtained from each subject about the duration 
of her menstrual cycle and the point in the cycle at which she then 
was. Arrangements were then made for each subject to take the 
MMPI on the second day of menstruation, the eighth day in her 
menstrual cycle, the fifteenth day in the cycle, the twenty-second 
day in the cycle, and two days before menstruation (Test A, B, 
C, D, and E, respectively). 

This schedule was varied a bit depending upon the actual length 
of the cycle and to avoid testing on Sundays. Each subject took her 
first MMPI (Test 1) at the first test point in her menstrual cycle 
which came up subsequent to the interview. One subject took her 
first MMPI at point A (see schedule above), two took their first 
MMPI at point B, three at point C, one at point D, and three at 
point E. For the group, Test 1 and Test 2 were separated by a mean 
of 5.3 days (range 3-7 days), Test 2 and Test 3 by a mean of 5.5 
days (range 4-8 days), Test 3 and Test 4 by a mean of 5.5 days 
(range 2-9 days), and Test 4 and Test 5 by a mean of 5.4 days 
(range 3-7 days). 

An IBM card form of the MMPI was used and was scored by an 
IBM 101. In a few instances (as when a subject had to be away 
from school on a test day), the booklet form was used. 

r In onder to promote the most open response to the test, each sub- 
ject was given a code number; the number was the only identifica- 
tion which went on each test, and the test results were filed until the 
end of the study at which time each subject was paid fifteen dollars 


and a card-bearing the subject’s name and code number was de- 
stroyed. 


Results 


Table 1 presents the mean T-scores (K-corrected) of each scale 
for each testing (from the first test, regardless of point in menstrual 
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cycle, to the fifth test). For most of the scales, the change in mean 
T-score from testing to testing is under 3.0 points; in only three 
instances is this exceeded (an increase of 4.2 on scale 3 from Test 
2 to Test 3, a decrease of 3.8 on scale 4 from Test 1 to Test 2, and 
an increase of 5.6 on scale 5 fromTest 2 to Test 3). The median 
change is 1.2 T-score points. 

Analysis of variance in a treatments by subjects design (Lind- 
quist, 1953) was used to test the hypothesis that there are no differ- 
ences between the groups. Using the mean squares for testings 
(df = 4) and for treatments by subjects (df = 36) as the numera- 
tors and denominators, respectively, the F-ratios listed in Table 2 
were computed. None of the F-ratios was significant at the .05 level. 

Discussion 

The results of the analyses of variance show that the observed 
changes in mean T-scores from testing to testing could be expected 
to occur often enough on the basis of chance that one would want 
to be cautious in attributing them to any systematically inter- 
vening factor. The mean changes in themselves, even taken at face 
value, are not large. 

The reason for lack of change is not readily apparent from this 
study. It could be due to a lack of significant change in the sub- 
jects, this subject stability being accurately reflected in the MMPI 


TABLE 1 
Mean T-Scores (Ten Subjects) of 15 MMPI Scales for Each of Five Testings 


Test 2 Test 3 Test 4 Test 5 


Scale L 44.9 45.0 45.1 45.2 

Scale F 51.7 49.9 50.4 48.7 40.9 
Scale K 54.0 55.0 57.0 os as 
Scale 1 (Hs) 48.6 47.9 48.0 40,9 ep 
Scale 2 (D) 50.0 48.0 46.0 na A 
Scale 3 (Hy) 57.0 57.6 53.4 53.8 66.1 
Scale 4 (Pd) 56.5 52.7 53.9 HA no 
Scale 5 (Mf) 41.8 41.9 41.8 45.0 43.5 
Scale 6 (Pa) 56.6 58.9 a oe Be 
Scale 7 (Pt) 55.7 55.5 non ee 4 
Scale 8 (Sc) 58.2 55.3 58. ps 50.5 
Scale 9 (Ma) 62.6 61.2 al 9 ae 18 
Scale 0 (Si) 49.1 49.7 3 os aa 
Scale A 50.7 51.4 49.3 a p 
Scale R 48.8 48.5 48.6 49. am 

— uainted Th the MMPI, a brief description of the test and of eae 
may ba iaaa D ATAA MoKinley (1951). The A and R scales are described in Dahlstrom 


and Welsh (1960). 
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TABLE 2 


F-Ratios Resulting from Analyses of Variance of T-Scores (K-Corrected) 
of 18 MMPI Scales Administered Five Times to Ten Subjects 


MMPI Scale F-Ratio* MMPI Scale F-Ratio* 

L 0.37 6 (Pa) 0.92 
F 1.07 7 (Pt) 0.25 
K 1.23 8 (Sc) 0.87 
1 (Hs) 0.20 9 (Ma) 0.17 
2 (D) 1.12 0 (Si) 1.33 
3 (Hy) 1.84 A 2.40 
4 (Pd) 1.29 R 0.42 
5 (Mf) 1.83 


VENone of the Frat h aaa Ge usss =P 
None of the F-ratios is significant at the .05 level @ of 2.69 required for 5 per cent level with 4 
‘and 80 degrees of freedom), Ey 


results. It could also be due to a lack of sensitivity to change, 
either because the MMPI does not do the job it should do (psy- 
chologists at Minnesota, would possibly resist such a considera- 
tion) or because the short intervals between testings permitted 
memory to play a role in determining responses to the items. 
Measures of the stability of individual MMPI scales and two- 
point codes (Dahlstrom and Welsh, 1960) and of MMPI profiles 
(Pauker, 1965, in press) have demonstrated respectable levels of 
retest reliability. The present study shows that repeated testing 
with the MMPI, at least up to five testings and with a young fe- 
male college group, has no marked effect on group scale means. 


Summary 


Ten young women, college students, took the MMPI five times 
during a one-month period. The data were grouped for each scale 
for each testing. Analysis of variance was done with the K-cor- 
eee T-scores of each of the three “validity” scales, the ten 
‘clinical” Seales, and the A and R scales. None of the resulting 
F-ratios was significant at the .05 level. 
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SHIFTS IN MEASURES OF ATTITUDES OF MEDICAL 
STUDENTS TOWARD THOSE OF THEIR PROFESSORS 
i RELATIVE TO THE DOCTOR IMAGE AND THE 
DOCTOR-PATIENT RELATIONSHIP: IMPLICATIONS 
FOR PREDICTION OF A CLINICALLY 
ORIENTED CRITERION 


SEYMOUR POLLACK: anp WILLIAM B, MICHAEL 
University of Southern California School of Medicine 
AND 
University of California, Santa Barbara 


In two previous studies, the writers (Pollack and Michael, 1965; 

1966) investigated changes in the attitudes of medical students to- 
ward psychological characteristics of the doctor image and the doc- 
tor-patient relationship on four empirically derived factor scales in 
Blum’s (1957) Patient Attitude Test (PAT)—three scales—and in 
the Doctor’s Opinion Questionnaire (DOQ)—one scale. Although 
only the mean changes on the dimension concerned with fee-setting 
practices could be judged as statistically significant, there were, at 
that time, no available data concerning the attitudes of doctors as 
measured on the three dimensions of the PAT, against which the 
attitudes of medical students could be compared. 


Purpose 


ratory investigation to determine 


It was the purpose of this explo y 
the extent and direction of shift in the standing of medical students 
al dimensions con- 


on each of twelve tentatively identified attitudin: ARYA 
cerned with the doctor image and the doctor-patient relationship 


z 366-0: the 
1 This research was supported by Research Grant No. MEOPTA TERE 


è National Institute of Mental Health to the University of Sou! 
School of Medicine. 


1069 


1070 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


toward the composite standing on the same dimensions of thirty-one 
clinical and thirty-one full time professors of a university medical 
school—dimensions which consisted of items from the PAT and the 
DOQ. The importance of such a pilot study was thought to lie in 
the tentative description of the nature of attitudes of medical stu- 
dents and in the determination of possible changes in the admit- 
tedly crude measures of those non-cognitive components that are 
seemingly inherent in any criterion of success in medical school. In 
other words, it appeared that in the prediction and evaluation of 
Success of medical school students—especially in clinical activities 
—any substantial shift of measurable levels of their attitudes in a 
direction toward those levels held by their professors could be of 
practical significance to a student’s success in clinical practice in 
that such success may depend, in large measure, upon a student's 
assimilating and accepting, or at least of giving the impression of 
assimilating and accepting, the attitudinal and value systems held 
by his professors. 


Procedure 


Items of the PAT and DOQ which met a criterion of endorsement 
by at least seventy-five percent of both the thirty-one clinical and 
thirty-one teaching staff at the University of Southern California 
Medical School (see Pollack and Michael (1966) for additional de- 
tails) were subjected to a preliminary content analysis involving 
four judges from whose pooled Judgments emerged twelve identifia- 
ble categories that are set forth primarily for their heuristic value 
in future scale construction. For samples of 59 freshmen, 55 sopho- 
mores, 40 juniors, and 33 seniors, who in early October, 1961 re- 
sponded to PAT and DOQ items placed in a questionnaire, the per- 
centages of their endorsements were calculated and compared with 
the standards established for the 62 professors. 


Results 


Tn Table 1, the twelve a priori categories are cited, along with 
median percentages of endo 


rsements of items in each category, by 
members of each of the four classes and by the sample of physician- 
teachers. Although the highly exploratory and preliminary nature 
of this investigation did not appear to justify the application of 
significance tests in the absence of cross-validation data, and de- 
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TABLE 1 


Median Percentages of Endorsement for Items in Each of Twelve 
` Response Categories Concerned with the Doctor Image 


and Doctor-Patient Relationship 
Doctors 
School Class — 
Numberof ——————___________ (Teaching and 
Response Category Items? Fresh. Soph. Jrs. Snrs. Clinical Staff) 

1. Rendering medical 

services fairly 6 74 80 75 78 88 
2. Feeling of competence 

in practice 13 87 89 85 91 95 
8. Fee-setting procedures 13 78 73 80 85 87 
4. Doctor as a person 5 83 91 91 91 86 
5. Medicine as a calling 2 84 79 86 85 83 
6. Professional ethics 6 71 74 73 78 91 
7. Doctor’s liability 5 73 71 75 86 89 
8. Organic (pathological) 

vs. patient orientation 23 73 78 78 85 84 


9. Self-confidence in 

medical practice 11 50 58 68 68 2 
10. Feelings of satisfaction : 

and ease in dealing 

with patient 4 77 84 91 91 oy 
11. Consideration of 

patient’s psychologi- 


12. cal needs 5 96 90 se %4 i 
Socializing with the 
patient 3 39 38 a) bd ss 


* Twenty-eight of the 95 items appeared in two or more categories. 


spite limitations which must be posed in terms of the possible opera- 
tion of response sets of acquiescence and social desirability in the 
respondents answering the question, inspection of the median per- 
centages tentatively suggests not only that the perceptions of med- 
ical students in each class are similar to those of the professors, but 
also that, in general, there are small shifts from year to year in a 
direction indicating a slightly closer correspondence of the percep- 
tions of students to those of their professors. Although the findings 
reveal that there is a rather close similarity between the preliminary 
measures of the attitudes of medical students and those of their 


professors—a correspondence which aaa pon o 
in their program—additiona z 
greater as students progress in Z rts are needed before 


ý tions involving numerous cross-validation effo 
: reliable and valid scales can be developed, not Ey e Dan, ae 
Roncognitive components for the clinically oriented criterion 
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were suggested by this pilot study but also to exhibit changes in the 
perceptions of medical students relative to those held by their pro- 
fessors. 
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Statistical Analysis in Psychology and Education by George A. 
Ferguson. New York: McGraw-Hill Book Co. 1966. Pp. x + 446. 

This second edition of Ferguson’s popular text follows the first 
edition by seven years. (See Binder’s review of the first edition in 
the Winter 1960 issue of this journal.) A considerable amount of 
new material has been included (approximately 60 pages exclusive 
of exercises) ; relatively little of the content of the first edition was 
revised. Those who were devotees of the 1959 edition of Statistical 
Analysis in Psychology and Education will probably find much to 
be pleased with in the 1966 edition. This competently done and 
comprehensive edition seems likely to capture an even wider au- 
dience than did the first. I 

The first eight chapters of the first and second editions of Sta- 
tistical Analysis in Psychology and Education are nearly identical. 
From Chapter 1 “Basic Ideas in Statistics” through the hard-core 
material on graphs, tables, measures of central tendency and vari- 
ability, probability, and the normal curve to Chapter 8 “Prediction 
in Relation to Correlation,” little has been changed. To be sure, 
some sections have been polished (the chapter on probability has 
been slightly improved), some have been eliminated (the geometric 
and harmonic means have disappeared and that vestige of a by- 
gone era, the bivariate table for the calculation of r, has been mer- 
cifully laid to rest), and the exercises, which were generally peor ji 
the first edition, are now passable. However, the bulk of the firs 
130 pages of both the first and second editions are about the ama 
Ferguson handles this elementary material competently, but wi Ei 
out particular distinction. The only substantial change Tom S ‘ 
first edition is in the definition of the sample variance. In me ire 
edition, s2 was the biased, maximum likelihood estimator of : ;in 
this edition, the unbiased estimator of o is used. This is clear! y an 
i l find the unbiased estimator diffi- 
improvement. Ferguson appears to the E anin 
cult to live with, however; in defining biserial and point- biseri ‘A 
Chapter 15 he reverts to the biased iene aed in an a 
tempt to keep the formulas compact and attracuve. 

The peal strengths of Statistical Analysis in Psychology and 


5 i i he best buy on the 
Education and the features which make it about t 
market are contained in chapters mne through twenty-four. Chap- 
ter 9 “Essential Ideas of Sampling” and Chapter 10 “Tests of Sig- 
nificance” of the first edition have become four chapters in the sec- 
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ond edition:“Sampling,” “Estimation,” “Tests of Significance: 
Means,” and “Tests of Significance: Other Statistics.” Although 
there is a certain logical elegance in introducing the standard er- 
tor of Z for finite populations first and then obtaining øs, for in- 
finite populations as a limiting case (as Ferguson has chosen to do 
in Chapter 9), this approach is particularly poor pedagogy. Given 
that the students have mastered the material in Chapter 7 on the 
variances of sums and differences of variables, comprehension of 
a proof that o,” = o,*/n should not be beyond them. Chapter 10 
“Estimation” is well done, though somewhat short. Ferguson is to 
be commended for having discussed the unbiasedness, consistency, 
efficiency, and sufficiency of estimators. Chapters 11 and 12 “Tests 
of Significance: Means” and “Tests of Significance: Other Statis- 
ties” are fairly complete and correct. The exposition of inferential 
statistics was improved in this edition; greater emphasis was 
placed on types of erros, levels of significance, and directional 
significance tests. Ferguson’s treatment of inferential statistics is 
preoccupied with tests of significance instead of interval estima- 
tion. The methods of setting confidence intervals around a statistic 
are given little emphasis or excluded altogether. Enough has been 
said elsewhere about the relative merits of these two inferential 
statistical approaches. Suffice it to say here that such reliance on 
tests of significance as is evident in Statistical Analysis in Psychol- 
ogy and Education is unhealthy. 

_ Chapter 13 “Chi Square” remains unchanged from the first edi- 
tion. Chapter 14 “Rank Correlation Methods” is actually poorer 
than it was in the first edition. The section of Kendall’s tau has been 
shortened to the point where it is difficult to imagine one learning 
how to compute tau from it. Ferguson added a section on rank cor- 
relation When one variable is a dichotomy, but he chose to present 
a Coefficient due to Whitfield instead of Cureton’s rank biserial co- 
efficient, which is probably a superior measure. When Chapter 15 

Other Varieties of Correlation” is coupled with the other chapters 
on rank correlation methods and Pearson’s r, few correlational 
measures of importance are missing from the text. The sixteenth 
chapter “Transformations: Their Nature and Purpose” is unique. 
Ferguson has drawn together and unified standard scores, percen- 
tile points and ranks, normalizing transformations, the stanine 
scale, regression transformations, and transformations with age al- 
lowances. Though it might be argued that this chapter belongs 
nearer the front of the text, there is no doubt that it is superbly 
written and a valuable inclusion. 

Chapters 17 through 20 comprise a first-rate primer in experi- 
mental design and analysis. The first chapter of this set, “The 
Structure and Planning of Experiments,” is new to the book; in- 
deed it is an innovation in elementary statistics instruction. The 
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objectives of the chapter are modest (only basic terminology, ran- 
domization, etc., are discussed), but it sets the stage well for chap- 
ters 18 and 19 in which one-way and two-way analysis of variance 
procedures are presented. Ferguson’s presentation of the analysis of 
variance is not uniformly good. The statistical assumptions of the 
technique are handled poorly, being stated imprecisely and incom- 
pletely near the end of the discussion of one-factor designs. The 
independence assumption in the analysis of variance is omitted 
from the list of assumptions on page 294. This is a pardonable 
oversight, but Ferguson compounds the omission into a mortal sin 
in the two-way analysis of variance chapter by analyzing two 
dependent-groups (repeated measures) designs as though an inde- 
pendent-groups analysis were appropriate. The entire discussion 
of the analysis of variance could be improved substantially by 
greater care in treating assumptions. In repeated measures de- 
signs, F-tests require spcial modifications; but Ferguson gives no 
indication of this, On page 278, he points out the virtues of the 
repeated measures design but never mentions how to deal with its 
special problems. The effects of violations of assumptions on prob- 
abilities of Type I and Type II errors have now been thoroughly 
charted by mathematical statisticians. The importance of these 
problems and the success of recent attacks on them are not re- 
flected in the cursory treatment given them on pages 294-295. 
Ferguson revised a dismal section in the first edition on following 
the F-test with multiple t-tests; the corresponding section in the 
second edition is a brief treatment of Scheffe’s multiple comparison 
procedure, The inferential pattern of Scheffe’s procedure (error 
rates are “experiment-wise” rather than “test-wise”) is not dis- 
cussed sufficiently, and Tukey’s multiple comparison procedure (su- 
perior to Scheffe’s for simple contrasts) is conspicuously absent, 
Chapter 20 “Analysis of Covariance” and Chapter 21 ‘Trend 
Analysis” are valuable additions to the second edition. Very little 
of the rationale of analysis of covariance is given; but the impact 
of the chapter on the student is unmistakable: he should be na 
perform a one-way analysis of covariance and point Es 
homogeneity of regression coefficients assumption, as well, nr z 
reading the chapter. One can ask for more, but he seldom gets as 
much. The chapter on trend analysis will þe out of reach for many 
students. he 
The final three chapters of Statistical Analysis in Psychology 
and Education are essentially unchanged from the first edition. 


“Selected Nonparametric Tests” does include two sections on non- 


i i i be original contribu- 
parametric trend analysis which appear to 
tions by Terni This chapter is marred. only by the absence of 
sufficiently explicit statements of the statistical hypotheses being 


tested. 
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One really has no right to insist that a book be compendious, cor- 
rect, and well written too; but an easy, sometimes humorous style 
is an added bonus. Ferguson’s prose tends to be dry and stolid and, 
infrequently, annoying (e.g., the repeated use of “comprised of”) 
The gremlins which plague statistics textbook writers left evidence 
of their work at several points in this second edition. Perhaps the 
most embarrassing error to the publishers is the misspelling of the 
name of the consulting editor for statistics (Jonas for Jones). The 
scatter diagram in Figure 8.1 is inaccurately drawn. The expecta- 
tion of the F-ratio is not unity given a true null hypothesis, as is 
claimed on page 319. (The expected value of Fn no iS n2/ (n2 — 2).) 
The correlation coefficient is said to equal the geometric mean of 
bye and bey, but the geometric mean is not defined for negative num- 
bers. A question-begging proof that r cannot exceed -+1 or be less 
than —1 is given on page 109 (How does one know that r is maxi- 
mal when the z-scores on x and y are equal for all persons?). We 
are inclined to disbelieve his complicated expression for the ex- 
pected value of the mean-square “between” in the one-way analysis 
of variance (p. 288); the sole expression for E(MSper,) which we 
can justify, namely o? + 3nja?/(J — 1), equals Ferguson’s expres- 
sion only when all n; are equal. That Ferguson’s formula is equiva- 
lent to the expression for E(MSvet,) in the random model for un- 
equal n’s leads us to believe that he drew an incorrect analogy 
between the fixed and random analysis of variance models at this 
point. Regardless of how the error arose, his formula should be dis- 
regarded. The statistical faux pas are far less frequent in Ferguson's 
bis ean seals works. The technical accuracy which Bin- 
Pai ommended in the first edition of Statistical Analysis in 

‘ychology and Education is evident in the second edition. Fergu- 
son will seldom lead the reader into error. 
aoe š (1960) concluded that the first edition of Statistical 
pe in Psychology and Education was “wholly adequate and, 
ae ue eee text.” The revisions and presentations of new 
Fa en the second edition make the text clearly super- 

petitors. Its continued popularity seems assured. 
Gene V Grass anp THomAs O. MAGUIRE 
University of Illinois 


Ferguson, George A. Statistical Analysis in $ oni lu- 
cation. New York: McGraw-Hill Book Go. 1959. Eae 


BOOK REVIEWS 1079 


Analysis of Variance by William C. Guenther. Englewood Cliffs, 
N. J.: Prentice-Hall, Inc., 1964. Pp. viii-199. $7.95. 

This book was written by a mathematical statistician for stu- 
dents with limited mathematics background. It assumes a semester 
of both pre-calculus college mathematics and statistics. The book is 
divided into six sections. The first section is a 29-page review of 
statistical inference. Included in this review is a discussion of t- 
tests, tests for homogeneity of variance, and a number of basic 
statistical definitions such as Type I and Type II errors and the 
power of statistical tests. The next four sections of the book are con- 
cerned with a number of simple experimental designs such as the 
simple one-way analysis of variance, random blocks, latin squares, 
and factorial designs. The sixth and last section of the book dis- 
cusses analysis of covariance. 

An attraction of the book is that many aspects of analysis of 
variance are discussed which are not usually covered in some of the 
traditional texts written primarily for psychology and education 
students. The book itself is not slanted toward any particular 
audience. The examples are taken from all disciplines. The au- 
thor spends considerable space in partitioning the total sum of 
squares for each of the designs. He also discusses the use of ortho- 
gonal contrasts for each of the designs and emphasizes the use of 
multiple comparisons for arriving at the significance level of con- 
trasts. Of particular importance is the author's discussion of the 
power of the F tests for each of the experimental designs. Most 
psychological and educational statistics textbooks devote little or 
no attention to power calculations or to a consideration of the 
power of the various F tests before the experiment is begun. 

The most attractive feature of the book is the differentiation be- 
tween fixed and random effect models. The author discusses the ex- 
pected mean squares for both cases and points out the differences in 
the F ratios between the two models. The concept of interaction i 
given good coverage. The assumptions for each design are aa 
spelled out, and the importance of randomization 18 ke ad 
emphasized. Other useful techniques m e for nonadditivity 
and i lations are also covered briely. — $ 

e e of the book is that it is ee P 
concisely with brief discussions of a large number 0 ST s. 
Some of the formulas look rather formidable and this mig 

by itself would probably not suf- 
courage some students. The book by 1 i Asiana 
fice as a text in experimental design for psychology an i T 
students because of its consciseness and limited coverage. However, 


in emphasizing some of the points 


i i atment. 
texts give little or no tre: Gnoran H. DUNTAMAN, 


University of Florida, Gainesville 
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Nonparametric Trend Analysis (A Practiced Guide for Research 
Workers) by G. A. Ferguson. Montreal: McGill University 
Press, 1965. Pp. 61. $1.95, paper. 

In a fascinating little monograph—albeit a rather expensive one 
—Ferguson has created a guidepost by which the practical re- 
searcher may lead himself through the paths of nonparametric 
analysis for trend. 

Ferguson needs no introduction as a lucid, careful writer. In the 
present booklet he does nothing to destroy that image—in fact, 
this is Ferguson at his clearest. For the most part, the printers have 
done an excellent job for him. The shape of the monograph is a 
trifle awkward perhaps, but the greatest drawback in format is the 
inclusion of tabular reference material in the body of the text. Hav- 
ing to hunt for a table is always time-consuming, at best. One final 
comment on the format—the binding would appear to be inappropri- 
ate to the kind of “recipe-book” function the monograph sets it- 
self, The review copy was—after two mailings—already in a state 
of dismemberment. Presumably the book would not last the user 
who attempted to open it fully, much longer, It is a pity that a book 
with such a heavy reference-orientation cannot be bound to make 
it easier for desk use, 

Nonparametric trend analysis parallels, for ordinal data, the 
parametric technique of orthogonal polynomials. That is, it is a 
method designed to facilitate the study of the relation between 
treatment and experimental variables to see whether it is monoton- 
ically Mereasing or decreasing, and what the general shape of the 
functional relationship is. As with other nonparametric techniques, 
the methods described in the monograph void themselves of the 
usual assumptions of the analysis of variance model which they 
Teplace—i.e., no assumptions of normality or of homogeneity are 
made. Because of the nature of the data on which it relies, non- 
a h analysis is potentially highly appropriate for the 
hee a pa monograph presents a complete method for con- 

g the analysis. Basically, it is an extended presentation of an 


means that the ranks of the second set tend to i icall 
with an increase in the naturally-ordered ee 
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dicates that the ranks of the second set show a tendency to decrease 
monotonically. § is a particularly appropriate statistic—its asymp- 
totic normality and its power, were investigated by Mann twenty- 
one years ago. Problems involving ties have, in general, been fully 
explored. And one of the most powerful nonparametric tests, the 
Mann-Whitney-Wilcoxon U is yet another application of S. 
Chapters 1-3 in the monograph (covering the first 18 pages, or 
30 per cent of the space) are introductory. Chapter 1, in fact, de- 
scribes how the monograph was written in response to some prob- 
lems encountered in making an analysis of data from experiments 
conducted in the Department of Psychology at McGill University. 
Chapter 2 defines the terms used, and Chapter 3 summarizes the 
computational procedures for S and for the variance of its dis- 
tribution. In Chapter 3, the author caters to the researcher by end- 
ing the section, and footnoting each table, with sets of rules. Some 
of the phraseology in these rules is perhaps rather terse for a non- 
statistician: e.g., “when one variable is a dichotomy and the other 
contains m groupings of values of extent t where t = 1, reduce the 
absolute value of S by the correction . . .” (p. 18). One can imagine 
an unsophisticated researcher looking for a set of simple directions, 
being unwilling to read through the text, and being more than a little 
bewildered at the technicality of such language. A 
Chapters 4-6 set out the tests for monotonic trend with correlated 
data. Chapter 4 is concerned with large-sample procedure without 
ties. The suggested method is an alternative to the Friedman 
two-way analysis of variance, by replacing the original D 
ments with ranks. Chapter 5 treats the case of large samples te 4 
ties; Chapter 6, small samples. Chapters 4 and 5 E TAR a 
examples laid out as tables, but which merit only rather aii 
bowelled mention in the text. It could be helpful to know es le 
data are—to give the EES ae ae kinds of vari- 
ables for which the analysis has been found heipi. ‘ 
Chapter 7 sets out ite procedures for testing for 2 oa la 
in independent samples. $ is calculated in the usual way fro: Soa. 
data set out in a manner similar to a Kruskal-Wallis aga A 
ysis of variance, and using a normal score ne y SRR 
reader who is familiar with any compendium o erage Weert 
tests, will be appreciating the unification brought al T em 3 
8. It is not always a opi Se ge 
many familiar tests is brought out very aly: i 
; v ae 5 returns to a theoretic discussion of ee ts 
nomials and the concept of nonmonotonie rank corre’ i a kA 
latter, ranks of a set Y are compared with the natura ee of 
ranks on a set X. Monotonic rank correlation is, ee care 
the disarray of Y in relation to the ranks for a first-degree Poe 
onal polynomial. The possibility op extension 
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and higher-order polynomials is considered. An extensive Chapter 
9 works through illustrative examples in the case of correlated 
data; Chapter 10, for independent samples. 

Here, then, is a satisfying monograph bringing together under 
one statistic a number of techniques already available for the most 
part, but perhaps not so well set out for the practical researcher. It 
is to this audience that one commends the book. And certainly it 
deserves a corner in the bookshelf of statisticians in the areas of 
psychology and education. 

PETER A, TAYLOR 
Rutgers University 


Readings in Educational and Psychological Measurement by Clin- 
ton I. Chase and H. Glenn Ludlow (Editors). Boston: Hough- 
ton Mifflin Co., 1966. Pp. xi -+ 381. $4.25 

Instructors of courses in measurement and evaluation are fortu- 
nate in having a number and variety of rather good texts on 
which to draw. However, instructors do have a problem in finding 
ways of supplementing the basic text. One can assign students to 
read articles in the various journals, but this creates the scene of 
students falling over one another to obtain the library’s one copy 
of the particular journal. The usual upshot is that the journal mys- 
teriously disappears or someone discovers that the article was 
neatly removed with a razor blade. A second alternative is to pre- 
Pare mimeographed copies of supplemental readings. This is fine 
from the student’s point of view, but requires a tremendous amount 
ee time—a commodity in rather short supply in most 

ces. 

Chase and Ludlow, responding to a real need, have prepared a 
paperback book of supplemental readings. The book consists of 
forty-four articles, research reports, and “pieces,” which should go 
a long way towards satisfying most instructors and, hopefully, 
students. It is a delicious collection, and most instructors will be 


i k seven units, but this arrangement 
is relatively unimportant, It is the articles themselves that are im- 


present a partial list of the contents. These include: Tyler’s “Gen- 


igs ‘ Cronbach and Meehl’s “ truct 
Validity in Psychological Tests,” ee “Problems ae De- 


K onzik, Macfarl. d Allen’s “The 
Stability of Mental Test Performance Biv Two and Eighteen 
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Years,” Coleman and Cureton’s “Intelligence and Achievement: 
The ‘Jangle Fallacy’ Again,” MeNemar’s “Lost: Our Intelligence. 
Why?,” Wesman’s “Faking Personality Test Scores in a Simulated 
Employment Situation,” and Super’s “The Multifactor Tests: Sum- 
ming Up.” There are many other equally fine articles. 

The only objection this reviewer would raise, albeit a minor one, 
is the lack of historical material. For example, inclusion of some of 
Binet’s, Ben Wood’s and E. L. Thorndike’s writings could have 
added an important historical perspective for the student. However, 
treat this objection lightly. The collection is first rate and deserves 
thoughtful consideration by every instructor of courses in evalu- 
ation and measurement. 

RicHarp WoLF 
University of Southern California 


A Practical Introduction to Measurement and Evaluation (Second 
Edition) by H. H. Remmers, N. L. Gage, and J. Francis Rum- 
mel. New York, N. Y.: Harper and Row, 1965. Pp. xvii +390. 

From their acquaintance with previous works of the authors, the 
reviewers had higher expectations for this book than were realized. 

The second edition differs little from the earlier version; brief sec- 

tions on item analysis, the affective domain, and the standard error 

of measurement constitute the major changes. A 

Although billed as a “practical introduction,” the book evidences 
no particular strength in this respect. It appears to be overly am- 
bitious in scope, touching topics which range from measurement 
theory through mental hygiene. As a result, several important con- 
cepts are either minimized or over-simplified. = rated? 

Too frequently the reader is given an admonition wit Wi co 
crete assistance as to how to carry it out. Note the following 

illustrations: il a 
Pupil evaluation should be used as a basis for improving pubio 
relatis and the mobilization of public opinion (pp. 11-12); 


i ide the decision 
... the teacher’s knowledge of the pupil should gui : 
of the pupil and his parents regarding the type of secondary ed: 


ucation to choose. (p. 91) ; 
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tioned in the list of typical achievement tests; NCME is presented 
as the former NCMUE; the Journal of Educational Measurement 
is not included along with six other journals listed as carrying re- 
ports of research with tests; some of the suggested references ap- 
pearing at the end of each chapter are not the most recent editions; 
and finally, the obsolete probable error is defined and mentioned 
several times throughout the book. These examples, along with the 
general tone of the presentation, serve to indicate a lack of care in 
updating in content and emphasis. 

Chapter 2, which presents an introductory survey, unfortunately 
becomes an extended glossary of types of tests rather than a gen- 
eral orientation to measurement and evaluation. This feature may 
have an initial negative impact upon the reader. The chapter, “Sta- 
tistical Concepts Used in Measurement,” although comprehensive, 
is not organized in a highly teachable manner. The care taken in 
presenting steps for grouping data to facilitate the computation of 
the standard deviation is somewhat inconsistent with the raw-score 
approach used for determining a coefficient of correlation. Too much 
information of a tangential relevance may hinder the students’ ac- 
quisition of the major concepts; for example, three different meth- 
ods for determining split-half reliability are given. Certain points 
are either imprecisely or incompletely made. For example: T-scores 
and stanines are not presented as being normalized; ordinate is 
used where abscissa is implied (p. 56); no mention is made of the 
mappropriateness of using the Spearman-Brown formula with 
speeded tests. Two scatter plots illustrating correlation are im- 
properly scaled; the range on one axis is about four times as great 
as on the other, There is no mention of the widely used deviation 
1Q. A more serious error appears in the definition of a “true score” 
which is said to be “. .. a score that would be obtained by an ideal, 
infallibly accurate Measurement.” (p. 134). The definition obscures 
the very important point that constant error is an ingredient of a 

true score, Further, there is an unfortunate promulgation of the 
Widespread misconception that a test, in order to be maximally 
efficient, needs a wide range of item difficulty (p. 203). 

RAN sections on validity and reliability are generally good. 
PARS eut are apt to lead the beginner astray: (a) 
(Hike erates prectia as a synonym for content validity; 
S is a ility can be increased at the expense of va- 
thi e extent that the student may perseverate on 
ka oe rather than on the more typical pattern, namely, that 
a fees eea a (c) the multiple-choice test is stated 
PARE, sets, ignoring its vulnerability to the po- 
cne gambling” and “speed ys. accuracy” response styles; (d) the 
diagnostic value of profiles is both implicitly negated (p. 112) and 
supported (p. 113); and (e) classroom teachers are told that they 
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can standardize their tests by applying statistics (p. 95). Unfor- 
tunately, little attention is given to the meaning and interpretation 
of reliability and validity data. 

The chapter on Test Administration and Scoring is very good, 
although a clearer distinction between suggested teacher-behaviors 
for standardized versus teacher-made testing could profitably have 
been made. 

The main strength of the book is found in Part Three: The Eval- 
uation of Classroom Instruction, in which a good discussion of 
evaluation in relation to necessary consideration of educational 
objectives is presented. This section, unlike the others, is supported 
by relevant documentation and useful illustrations. The relative 
merits of different approaches to testing, for example, objective 
versus essay, provide practical guidelines for improvement of 
teacher-made tests. It provides a good source of examples for con- 
struction of various types of test items. The attempt to deal with 
the complexities of assigning marks and reporting pupil progress 
adds little new insight to an everpresent problem. 

Part Four: Appraisal of Personality Aspects is a mixture of 
mental hygiene vocabulary and concepts and an inadequate and 
uncritical treatment of interest and personality assessment. Cur- 
iously, the Vocational Interest Inventory and the Occupational 
Interest Inventory are given as much emphasis as the Kuder and 
the Strong. The Allport-Vernon Study of Values is presented as in- 
terest inventory, It is unfortunate that the authors recommend ad- 
justment inventories for general pupil use. Thus, they give inad- 
equate attention to the problem of validity and level of training 
which is necessary to properly utilize such instruments. ; 

The reviewers regret that the attention, care, and quality which 
the authors have manifested elsewhere in their work was not re- 
flected in the present effort. 

Kennetu D. Hopxrns AnD WILLIAM A. Suse 
University of Colorado 


Building a School Testing Program by Robert H. Bauernfeind. 
Boston, Mass.: Houghton Mifflin Company, 1963. Pp. xvii + 
343. $5.25. 

This text is designed specifically for school people who are re- 
sponsible for deciding which tests to use at which grade levels. 
Thus, the book is intended primarily for school administrators, 
testing directors, and for graduate students in programs that may 
lead to their becoming school testing directors. Although the author 
assumes that the reader has a basic knowledge of educational 
measurements, he takes the position that statistical methods are 
not crucial to the work of school testing directors. The result is a 
text which recognizes the importance of the statistical concepts 
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inherent in measurement, but treats them conceptually rather than 
computationally. 

The text is organized in three major parts. Part I emphasizes the 
basic concepts of test scores, test norms, correlation, validity, and 
reliability. Although the overall discussion of the concepts lacks 
depth, Bauernfeind does an exceptionally able job of providing the 
essentials needed for use in test selection and implementation. In 
regard to establishing local norms, this is one of the few texts 
which advises strongly for the use of stanines over other methods of 
converted scores. The discussion of the advantages and disadvan- 
tages of using stanines which is done quite well should be under- 
standable even for an unsophisticated school testing director. 

Part II is clearly the strongest aspect of the text and perhaps the 
most useable for those who are responsible for initiating a testing 
program. There are chapters on testing in grades kindergarten 
through three, testing for educational development and for mental 
ability in grades three through twelve, testing for vocational apti- 
tudes and vocational interests in grades eight through twelve, 
measuring personality characteristics, and using subject-matter 
achievement tests. In this part especially, Bauernfeind does not 
hesitate to offer specific advice and guides to action. As a whole, 
the suggestions are sound, and a school testing director would be 
unlikely to go wrong if he were to follow them without question. 
However, Bauernfeind’s willingness to prescribe may occasionally 
place a competent course instructor in an awkward situation, since 
for various reasons the instructor may at times prefer alternate or 
different emphases from those prescribed in the text. However, the 
flavor of the volume makes this event unlikely. A potential user of 
the text would, of course, have this problem resolved prior to re- 
quiring the book for course use. As is the risk in using any text, if 
the instructor raises too many questions, he may undermine the 
students’ confidence in the text, himself, or testing in general. Thus, 
for the most part, if a book is used, he will have to join with 
Bauernfeind in most of the suggestions given. 

A concluding chapter illustrates an example of how one school’s 
testing program was revised through using the recommendations 
and Procedures emphasized in the preceding chapters. Actually, 
the material which preceded this chapter provides all the cues and 
pei it necessary for establishing an effective testing program 
he pest make the chapter redundant. A greater detriment may 
tha hapa al testing director will go directly to 
gredients Re in the eer ee the oe 
F sting DORAO rei preceding chapters to generate his own 

unique inclusion in the text is a section in which nine leading 
test specialists attempt to forecast future developments in educa- 
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tional testing. Although the forecasts are provocative, they add 
little to the immediate objectives of the book. However, they do 
tend to illuminate trends and issues to be concerned about in the 
overall domain of educational testing. 

Noticeably lacking in the text is any functional information on 
how data processing techniques might be implemented in building 
a school testing program. Unfortunately, the author dwells only 
passively with the subject and even then in an admittedly sus- 
picious fashion, This gap is even more pronounced when one reads 
of the importance attached to data processing by each of the nine 
contributing test specialists. 

Overall, the text is functional, and well presented. As Bauern- 
feind acknowledges, some measurement specialists may have res- 
ervations about the nonstatistical nature of the book and about 
some problems and concepts that have been unduly simplified. 
However, there is little evidence that highly sophisticated texts 
attempting to do the same thing have had any great observable im- 
pact on “building a school testing program.” Thus, it is the impres- 
sion of this reviewer that the shortcomings cited are well out- 
numbered by the strong points. 

James C. Moorn 
The University of New Mexico 


Individual Differences by Anne Anastasi (Editor). New York: 
John Wiley & Sons, Inc., 1965. Pp xiv + 301. 

This excellent little volume affords the measurement expert the 
opportunity to sample nearly a century of the most pertinent pub- 
lications in the field, from Galton to Guilford. The thirty titles are 
interspersed with editorial passages which do just what Anastasi 
hoped for. They act “as a guide who takes the reader on a personally 
conducted tour of the literature.” 

Individual Differences will prove a valuable book for teachers, 
because it provides materials which are not readily available in 
many libraries (e.g., three of the Binet and Simon articles and sev- 
eral Galton works). This reviewer observed that almost all the 
works have been listed by Anastasi as references in her earlier dif- 
ferential psychology text; she must have made her choices of these 
particular entries partly because they have enabled her during her 
years of teaching to best’ put “the psychological study of individual 
differences in historical perspective.” 

The first chapter is a succinct survey of that which went before 
and led up to the scientific study of individual differences. Hach of 
the following chapters contains from two to four chronologically 
arranged and annotated readings on ten different topics, concern- 
ing or closely related to measurement. The ten topics are: Measure- 
ment of Individual Differences, Nature of Intelligence: Pioneer 
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Research, Nature of Intelligence: Later Developments, Behavior 
Genetics: Statistical Studies, Behavior Genetics: Theoretical Ori- 
entation, Cultural Deprivation: Cross-Sectional Approaches, Cul- 
tural Deprivation: Longitudinal Approaches, Nature of Genius: — 
Early Exploration, and Nature of Genius: Focus on Creativity, — 

The concluding chapter briefly gives an overview of the selections 
as they relate to the story of the progress and expansion of knowl- 
edge about individual differences. The editor points out that some 
of the selections were chosen to demonstrate that false starts have 
been made and that the “history of differential psychology has 
been particularly productive of controversy.” 

All in all, students and teachers of measurement as well as re- 
searchers with an appreciation of history’s impact on the present 
should find this a valuable volume to own. The selections are well 
chosen. They are not so highly technical that they cannot be read 
with ease; they are interesting; and the editor’s comments tie the 
selections together so as to lend a certain charm often found miss- 
ing in scientific writing. 

Sister M. JACINTA MANN 
Seton Hill College 


The Psychology of Individual Differences (Third Edition) by 
Leona Tyler, New York: Appleton-Century-Crofts, Inc., 1965. 
Pp. x + 572. 

Once again Leona Tyler has made a theoretical and practical 
contribution to the area of individual differences. In her usual clear, 
interesting, and informative style, she has managed to synthesize 
the overwhelming amount of material in the area into an extemely 
readable and carefully organized textbook. 

€ years have elapsed since the second edition of her book. 

The format of the third edition is similar to that of the first two, 

but much updated. Again, as in the second edition, there is a lesser 

Riran on the presentation of statistical concepts than there 

1 in the original textbook. The concepts presented form an 
ies Pen the text. They are not an added appendage. The 

Fp ase As in which Part One proceeds from historical con- 

good. prekide general principles to research strategies provides a 

wine to the specific research findings presented in the later 


The separate presentation of individual i 
x and group differences 
pey for greater clarity. Presented in any other form, such numer- 
on bat could indeed be confusing to the student. As it stands, 
intita te the r a logically presented, but also provocative of 


This reviewer notes that an : 

; i y personal bias that the author 
pear have with regard to the findings is held in abeyance. Two 

pters seem particularly impressing and pertinent: Chapter 9 


b> 
is 


nr 
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on cognitive styles and Chapter 12 on race differences. The area of 
cognitive styles is very much to the fore at the present moment. 
This chapter gives a much-needed summary of the cognitive re- 
search to this point. The author sees the research of Witkin, Holtz- 
man, Klein and many others as occupying a “strategic position in 
differential psychology.” 

The chapter on race differences is especially good. The author 
neither wholeheartedly accepts the notion that one race is inferior 
to another, nor does she deny the empirical existence of such dif- 
ferences, She simply calls for research concerning the possible 
causes, Throughout the chapter, there is developed a real rationale 
for the organization of poverty programs, especially for the pro- 
grams which will reach the preschool culturally deprived young- 
sters. As she states in the Preface to the book, much of the ma- 
terial in the textbook should be related to the social reforms of our 
time. 

After treating individual and group differences, the author dis- 
cusses the factors which produce these differences. She dispels, once 
and for all, the idea of heredity or environment as the sole con- 
tributing factor. While treating each of these factors in separate 
chapters, she nevertheless maintains the necessary interaction be- 
tween the two. 

The final chapter “The Science of Human Individuality: Past 
and Future” gives a brief survey of some of the concepts which 
have arisen from the cited research, including the concepts of 
strategies and moderator variables. Appropriately enough, the 
chapter ends with a blueprint for the future. 

The Psychology of Individual Differences should prove helpful 
to the instructor, and enlightening to the student or proponent of 
differential psychology. This reviewer found it to be just this. 

Sister Mary Warrer HAUK 
Seton Hill College 


Two-Person Game Theory: The Essential Ideas by Anatol Ra- 
poport. Ann Arbor: The University of Michigan Press, Ann 
Arbor Science Paperbacks, 1966. Pp. 230. $1.95. 

As a technique that has considerable potential usefulness in the 
social sciences, decision-or game-theory has received relatively 
scant attention. Recent appearances—in increasing volume—of ar- 
ticles illustrating its applications, may hopefully imply that game 
theory is at last “catching on” amongst psychologists and educa- 
tionists. Within these two fields alone, the list of persons who could 
benefit from an acquaintance with game theory is too extensive to 
begin—anyone who has to make a decision in a conflict situation 
would warrant inclusion. It is pleasing to note, therefore, that this 
new little book, simple and readable, is available. 

Rapoport himself is a mathematical biologist. In preparing his 
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text, however, he has kept to a bare minimum any mathematical 
skills required to appreciate and to understand the content of the 
book. It is to this clarity and simplicity of expression that the book 
owes its values as a contribution to a widening of the social scien- 
tist’s horizons. For the most part, high school algebra, and a very 
meager knowledge of analytical geometry would more than suffice 
the reader. To be able to reduce what is a complex, and weighty 
field to this level of difficulty is in itself an achievement; and more- 
over to maintain interesting dialogue is an indicant of literary 
merit. 

The restriction on the mathematical sophistication demanded 
of the reader necessarily also restricts the scope of the book. The 
author has found it possible within the confines he has set to pre- 
sent the essential ideas of game-theory in the context of the two- 
person game. He leaves the general case for a later volume. By 
making this choice, notational simplicity is maximized, and at the 
Same time important topics may be treated in some depth. 

By a neat analogy, game theory is defined as being to games of 
strategy what probability theory is to games of chance, Game 
theory then, is not in the position to answer questions such as, 

What is the best way to play bridge?” but accepts questions of the 
Kind: “Ts there a best way to play bridge?” Game theory is, ac- 
cordingly, primarily concerned with the classification of games— 
and in this, has much in common with other classificatory sciences. 

Here, then, was a golden opportunity for the author to present 
to his audience—allegedly social scientists—examples and appli- 
cations in the fields of sociology, psychology, and education, for 
instance. Yet, few direct applications appear. Almost all the illus- 
trative examples in the text are confined to familiar games (such 
as tic-tac-toe), and to the eager social scientist, this lack of dem- 
onstration that game theory can be appropriately used in his field, 
may be something of a frustration. A 30-page chapter devoted to 
ois eas and limitations” does little to relieve that feeling. 

ure y, few amongst us can find solace in the assurance that two 
Petson game theory is highly applicable to war situations. 

n the positive side, however, one can find much to commend. 


ae 
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toe” and “square the diagonal” respectively. Although these chap- 
ters are clear and quite devoid of any but the most elementary 
mathematics, they become a little tedious—but this is probably the 
inevitable outcome of replacing mathematical symbols with words. 
Chapter 4 sets out the notions of the game tree and the game ma- 
trix; Chapter 5 discusses “dominating” strategies and the concept 
of minimax. A dominating strategy, sa, is, of course, one that will 
do at least as well as any other strategy, s,, no matter what 
strategy an opponent adopts. Minimax—relatively easy to explain 
mathematically—looks a little awesome in verbal garb, but the 
author handles the topic well. 

Chapter 7 discusses the two-person zero-sum game. In effect, 
the underlying theory here tells us that one is not attempting the 
impossible in looking for a best strategy for each player. The alge- 
bra is a little heavier in this chapter, but still about grade 10 level. 
Chapters 8 and 9 are particularly interesting. They discuss negoti- 
ated and non-negotiable games. The former of these (which arises 
when both parties hope to gain from the results) would seem to 
have particular application to, say, educational administration. 
The latter depends on a theoretical development which is quite sim- 
ilar to that of stochastic learning theories. 

Chapters 10 and 11 develop dynamic game models and in the 
process call on a smattering of differential calculus and analytic 
geometry. In a sense, these are the most enlightening chapters be- 
cause of their greater generality. 

Here, then, is a little book that many social scientists should be 
encouraged to read. It can scarcely be regarded as a text—only the 
most fortunate of universities at present have courses in education 
and psychology the content of which approaches that of this book. 
It is, then directed at an interested, semi-specialist audience. In 
providing the basic notions of two-person game theory, the author 
would seem to have achieved his objective admirably, and with re- 
markable clarity of expression. The book will not long appeal to 
those who are mathematically inclined—it is a litlte tedious for 
that. But then, it is not intended for the mathematician; and if it 
can whet the appetite of the administrator, the counselor, the clini- 
cian and the like, to seek further for possible applications of the 
method, the book will have made a major contribution. But, if those 
same people are looking for examples in this volume that are allied 
to their own field, they will be disappointed. 

PETER A. TAYLOR 
Rutgers University 


Pattern Recognition: Theory, Experiment, Computer Simulations, 
and Dynamic Models of Form Perception and Discovery by 
Leonard Uhr (Editor). New York: John Wiley & Sons, Inc. 
1966. Pp. xii + 393. 
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Pattern recognition has been an unpopular area of psychological — 


inquiry, but recent results from neurophysiology and from com- 
puter science have given new impetus to the area. Uhr has collected 
an interesting set of twenty-two readings that sample these new 
results as well as the philosophical, empirical, and theoretical back- 
ground of the problem. 

The selections have been sorted into five parts. Part I concerns 
the conceptual framework of the problem and contains contribu- 
tions of a philosophical nature by C. S. Peirce, E. Cassiere, K. Craik, 
and L. Wittgenstein. These selections contain material that is 
thought-provoking and fascinating, but taken as a whole they are 
fragmentary and do not provide a coherent, complete account of the 


and H. W. Hake reviews empirical work on form perception. The 
Hake selection is especially welcome, since it was previously avail- 
able only in a Wright Air Development technical report. Also in 


paper by L. G, Roberts provides an interesting insight into the 
bis advertised Perceptron, Three other ais recognition 
a ie are described by W. W. Bledsoe and I. Browning, by R. L. 

timsdale, F, H. Sumner, C. J. Tunis and T. Kilburn, and by Uhr 
ribes a general framework 
Viewed from historical per- 


| 
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Nor have the models described here made much contact with the 
neurophysiological findings of Part IV. Similarly, the simple con- 
tours that are generally used in the newer neurophysiological work 
are not much used in behavioral studies. Perhaps this interesting 
juxtaposition of the behavioral, neurophysiological, and computer 
simulation approaches to perception will encourage a closer rela- 
tionship in the future. 

The selections reflect, of course, the editor’s own point of view, 
which is that of a formal theorist with a preference for informa- 
tion processing models. Most of the selections show an interest in 
what information is transmitted by the sensory system and in how 
the system processes such information. But the selections are rel- 
evant and interesting from any point of view. 

Uhr remarks in the preface that pressure to keep the size of the 
book within some reasonalbe limit forced him to exclude many fine 
papers. Apparently, he also felt compelled to keep his editorial re- 
marks very brief. They are confined to only one or two pages at the 
beginning of each of the five parts. Otherwise, the papers are left to 
speak for themselves. Unfortunately, they do not speak in a co- 
ordinated way—the editor should have provided more setting for 
each paper. Also missing from the collection is any identification of 
the authors. Particularly in a multidisciplinary volume including 
philosophers, psychologists, neurophysiologists, computer scientists, 
engineers, and mathematicians, some background information on 
the participants would have been very helpful. The reviewer's final 
complaint is that in his brief commentary Uhr frequently compares 
or contrasts one of the included selections with some other work 
that was not included, the latter being identified only by some such 
phrase as “Steven’s work on specifying and programming Deutsch,” 
and there is no further reference anywhere in the book. Such name 
dropping may have its uses in face-to-face verbal interchange, but 
in a book it is an annoying discourtesy to the reader. 

Psychologists who are developing information-processing models 
of pattern recognition will find the book very useful. Interested 
psychologists will find it fascinating. Teachers may have difficulty 
using the collection in courses because considerable interpretation 
will be required to fit the articles into a coordinated framework. 
Nevertheless, by judicious selection many teachers will be able to 
use a significant set of the readings in a course in perception. 

Burt F. GREEN, JR. 
Center for Advanced Study 
in the Behavioral Sciences 


Computers and the Human Mind; An Introduction to Artificial In- 
telligence by Donald G. Fink. Garden City, N. Y.: Doubleday 
and Co., Inc., 1966. Pp. vii + 301. 

This book, one of the Science Study Series published by Anchor 
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Books, has two major objectives. The first is an introduction into 
the world of computers, and the second is an introduction into the 
field of artificial intelligence. It is not clear to this reviewer which 
objective is being sought by various parts of the volume. The Sci- 
ence Study Series is designed for the lay public (as indicated on page 
vii of this book), and therefore this volume must be analyzed ac- 
cording to its success in presenting, explaining, and perhaps con- 
vineing a lay (although intelligent) reader about computers and 
artificial intelligence. If a reader approaches Computers and the 
Human Mind with no prior experience on computers, and with per- 
haps limited mathematical background, he will be unable, and un- 
willing to find his way through the explanations offered. Although 
the book begins in a very elementary style, it soon presents exam- 
ples far more complicated. It would seem unlikely that a lay reader 
would be educated in computers in the first 17 pages sufficient 
enough to understand pattern recognition, and the immediate com- 
parison of mind to machine (the reviewer would prefer the word 
“brain”) in the first chapter is likely to alienate a considerable 
proportion of the population at the outset, 

There are a considerable number of lapses throughout the volume 
to language and mathematics for which the reader is offered no 
explanation. Examples of the derivation of m to 10,000 places, are 
not presented in a manner which would convince even the initiated 
as to why one would want to do such things. This book, therefore, 
1s not applicable as a means for explaining the functions and uses 
of computers to the public. 

With respect to its second criterion, that of artificial intelligence, 
the author seems to be more on his own ground. The presentation 
of human physiology and thought processes is quite well done for 
the type of audience likely to read it. The analogies to computer 


“thought” are less convincing, but those in the part on translation 
are well presented, 

The book suffers from too many styles, from a rather uninteresting 
Presentation of an intriguing and challenging subject, and from 
Poor organization. The contents could have been “programmed” 
more adequately. The volume presents too much information too 
quickly, and tries to serve too many masters. It is too elementary 


psychology course, and certain ch, i 
peda snes apters are useful for the specific 


Ricuarp E. SPENCER 
University of Illinois 
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The Tools of Social Science by John Madge. New York: Doubleday 
Anchor Books, 1965. Pp. ix + 362. $1.25. 

As Deputy Director of Political and Economic Planning in Eng- 
land, John Madge directs sociological and economic research for 
government programs, It is important for the reader to know that 
the author’s forte is sociology; the book’s title, however, demon- 
strates that the author adopts an interdisciplinary approach, Thus, 
his belief is supported that there are certain research tools common 
to all of the social sciences. The main social sciences cited in the 
book are sociology, anthropology, political science, and econom- 
ies. The author also considers history to be a social science. 

The specific social science tools discussed in separate chapters are 
documents, observation, the interview, and experiment. Although 
the author feels that all these tools are valuable, it is clear that they 
are ordered in such a way that experimentation is considered to be 
most desirable, i.e., the most distinguishing feature of a mature sci- 
ence. The problems connected with each technical research tool are 
discussed by the author. The first chapter of the book deals with 
the method of social science; special attention is given to the lan- 
guage and the logic employed in the social sciences. 

Since Sputnik I, institutions involved in teacher training have 
given a great deal of attention to the research methodologies of the 
various social science disciplines and of history. This emphasis 
has been especially true in the area of social studies education. 
How one knows what one knows is often considered as important 
or perhaps more important than what one knows. There is, there- 
fore, a need for books which focus on this subject. Although there 
are many criticisms which may be made against John Madge’s 
book, the book may prove to be of heuristic value to the reader 
interested in social science research methods. 

The most pointed criticism of the book is that it is dated: the 
most recent source cited is fifteen years old. The paperback edition 
has not been updated but is in the original 1953 form. Yet it is pre- 
cisely during the last fifteen years that there has been a surge of 
activity in questioning previously used social science research meth- 
ods and in forging ahead to develop new methods. 

It is also highly questionable that a sociologist can speak for social 
scientists in other disciplines as to research methods. This becomes 
increasingly true as each discipline becomes more sophisticated in 
developing methods of inquiry. Historians would probably be an- 
noyed at the author’s inclusion of their discipline in the social sci- 
ences. In short, the interdisciplinary approach adopted by the 
author should be carefully scrutinized by the reader. 

Finally, let it be said that the author may be labelled fittingly as a 
“traditionalist” in his approach to social science. That is, he does 
not support the “positivists”, or as he calls them, the naive be- 
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haviorists, who hold that all science is value-free. The reader may 
find the discussion of this subject as it appears in the book’s Intro- 
duction to be of more than passing interest, 
Darn L. BRUBAKER 
University of California 
Santa Barbara 


The Rand M: cNally Handbook of Education by Arthur W. Foshay 
(Editor). Chicago: Rand McNally and Company, 1963. Pp 294. 

As implied by the title, this is a reference-type publication offer- 
ing many principal facts about education in the United States. The 
major areas treated include (a) organization and administration, 
and (b) curriculum at the local, state, and national levels, In addi- 
tion, descriptions of the educational systems of England, France, 
and Russia are presented for the reader who has an interest in com- 
parative education, 

The volume provides huge quantities of factual information of- 
fered in an unbiased, well-organized, and succinct manner. It 
should provide a base from which anyone who has the interest 
could develop a fairly comprehensive overview of school adminis- 


Lye dread 
ther limitation of a work of this type i 
ner ype is the fact that many of 
the Statistics shown are outdated even before publication, At least a 
gc Te-editing effort will be Necessary to minimize obsoles- 
k: pokes rag Statistical data and Teanalyzing changing cur- 
The book is divided into three maj i ibuti 
0 aS aly. major parts with contributing 
rae ae in each area, assuming the major writing re- 
Part I, edited by Arnid J. Bueke, Di i 
Tna J, » Director of Studies, New York 
State Teacher Association, is devoted to organization and adminis- 
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cluded. Brief but effective statements regarding current issues with 
arguments pro and con, and additional references for further study 
are given. The editors have encouraged the reader to “whet his ap- 
petite and to go on from there,” 

Chapter 2, much of which is in tabular form, is concerned with a 
lengthy and technical description of all fifty states. Officials from 
each state were asked to check for correctness sections dealing with 
their respective states. Attention is given to a number of areas, 
ranging from constitutional provisions and legislative control to 
teacher certification and current issues in state school admin- 
istration. 

A discussion of local school systems with supporting data is pre- 
sented in Chapter 3. 

The major portion of Part II, Chapters 4 through 15, deals with 
the major subject matter areas which have traditionally consti- 
tuted the school curriculum. One chapter presents a brief discussion 
of curriculum methods and organizations in addition to some illus- 
trative daily school schedules. Attention is also given to current 
curriculum trends and issues. 

Chapters 17, 18, 19, 20, and 21 are concerned with audiovisual 
aids, textbooks, school libraries, adult education, guidance, and vo- 
cational education. 

In Chapter 22, several digests of controversial statements on edu- 
cational practices which offer promise for improvement are pre- 
sented. Concepts such as team teaching, independent study and 
ability grouping are given attention. 

According to the editor of Part III, Frank G. Jennings, editor at 
large of The Saturday Review, the section on education in England, 
France, and Russia is offered because of “greatly increased interest 
in the U. S. during recent years.” The contributing editor points out 
the difficulty in making any comparison of education in the United 
States with foreign countries and stresses that each educational sys- 
tem may best be presented in its own frame of reference. 

FRANK C. EMMERLING 

Comprehensive School Improvement Project 
Department of Public Instruction 

Raleigh, North Carolina 


Group Values through Children’s Drawings by Wayne Dennis. 
New York: John Wiley and Sons, Inc., 1966. Pp. xiii + 211. 
Readers of EPM are familiar with assessment procedures em- 
ploying drawings of human figures: Machover’s Draw-A-Person 
(DAP) and its predecessor, Goodenough’s Draw-A-Man (DAM) 
with a modern revision by Harris, and others. Dennis offers a third 
approach and bases its principles and purposes upon neither of the 
earlier drawing techniques. While Machover claimed to measure 
“global personality” and Goodenough measured intelligence, Dennis 
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stakes out a claim for measuring not the values of the drawer, but 
the values of the society to which the drawer belongs. Most readers 
will agree that the author’s claim which deserves careful considera- 
tion should not be lightly dismissed, 

In essence, Dennis makes only two assumptions in the application 
of his procedure. The first fits in well with projective theory: the 
drawn figure is derived from internal factors (values) rather than 
from figures observed in the environment. Considerable attention is 
paid to this point and the evidence, a mixture of observation and ex- 
perimentation appears to be ample. The second assumption con- 
tradicts the projective hypothesis. Dennis assumes that favorable 
attitudes are indicated for the figure drawn. Negative attitudes to- 
ward certain men (Dennis, like Goodenough, analyzes the 
drawing of the male) are indicated by one of two methods: (a) the 
absence of drawings of such men or (b) presence of selective distor- 
tion or ridicule. As might be suspected, the evidence presented on 
this assumption is not overwhelming, although it is reasonably con- 
vincing. The possibility that individual differences in drawing content 
might be important is considered in one sentence, However, to 
the reviewer, Dennis’ use of drawings to explore group values is 
probably on relatively firm ground without opening Pandora’s Box 


The technique is as simple as one has come to expect drawing 
techniques to be: a 846” X 11” sheet of paper, a number two pen- 
cil, and instructions to draw a “whole man.” The product is then 
analyzed into categories for which good inter-rater reliability is 
4 reasonable goal: (a) the presence of undesirable characteristics 
(facial scars, mouth corners down, crippled), (b) traditional vs. 
modern dress, (¢) emphasis on masculinity (facial hair, pipes, or 
Cigars), (d) smiling and nonsmiling faces, and others. The rele- 
vance of these items for gaining insight into the values of the group 
is discussed in brief detail. The groups presented for illustration 
are varied indeed: from the orthodox Hassidim in Brooklyn to mid- 
dle class boys in Heidelberg, to boys from a village in Cambodia, 
and to other Points in Mexico, Asia, Europe, and the Middle East. 

Wayne Dennis has Presented a provocative approach to inter- 


Pamir HIMELSTEIN 
Texas Western College of the 
University of Texas 


BOOK REVIEWS 1099 


The Results of Counseling by D. P. Campbell. Philadelphia: W. B. 
Saunders Co., 1965. Pp. 205. $6.00 

How many individuals secretly wonder whether counseling stu- 
dents really makes any difference in their lives? Here is a book, 
written in a concise and spirited manner which gives a clear affirm- 
ative answer. It deals deftly with a great deal of statistical 
information, enough to convince the most captious critic. And to 
frost the cake, it provides reassuring statistics on stability of 
testing, lack of decline in ability of this generation of college 
students as compared with those a quarter century ago, improve- 
ment of mental ability in adulthood, and sidelights on liberal-con- 
servative temperaments in our society. 

The basic material came from files in the counseling office at the 
University of Minnesota. Data were accumulated from 1933-36 for 
384 pairs of students matched on College Board Scores, English 
skills, age, sex, high school size, and class. In each pair one person 
was counseled and one was not counseled. In 1962, 731 of these 
former students were contacted and complete information was col- 
lected*on 80 per cent of them. Small but consistent differences in- 
dicate that over the years counseled students when compared with 
non-counseled ones are mildly less well satisfied with themselves, 
but have more accomplishments to their credit. They are less 
satisfied with their jobs, their marriages, their social life, and 
their over-all situation. But they have won more degrees, more 
honors, and more leadership posts. They make over a thousand dol- 
lars more per year, and score higher on a gross measure of con- 
tribution to society. It is truly surprising that differences such as 
these can be found after a lapse of a quarter century. 

The book contains an excellent chapter on a review of the litera- 
ture which focuses on motivated control groups, detailing the studies 
of Berdie, Jesness, Aldrich, Volsky and others, Hoyt, Vosbeck, Wil- 
liams, Golburgh, Rothney, and Schofield. This chapter admirably 
brings together material otherwise scattered and hard to find. 
Some of it involves high school counseling. 

The final chapters of the book explore three ancillary issues made 
available by the data. Hach issue is handled with much more detail 
than can possibly be discussed here. Nevertheless, a few remarks are 
in order. First, the data indicate that despite a much larger per- 
centage of the population attending college now, there has been no 
decline in mental ability when 1962 freshmen are compared with 
freshmen in the 1930’s. Second, it is shown that the former fresh- 
men, now adults, score higher on the same tests now than they did 
then—a finding indicating a continued rise in ability. Finally, there 
is a fascinating dichotomy of the group into liberals and conseryv- 
atives. The liberal is likely to be less religious, more unconven- 
tional, and have published more. The conservative is more likely 
to be a religious Protestant and to have a stable marriage. 
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The volume is well worth reading, for it raises some question 
while settling others. Is the “divine discontent” in the counsele 


up students to counsel than the stolid types? These are only a few 
of the questions which this volume raised for the reviewer. It will © 
pose some equally fascinating ones for others. The reviewer urges) 7 
the reader to obtain and to study this volume at his earliest con- À 
venience, 
Joun C. Gowan a 
San Fernando Valley State College” i 


The Outcomes of Counseling and Psychotherapy by T. Volsky, 7 
Thomas M. Magoon, Warren T. Norman, and Donald P. Hoyt, ` 
Minneapolis: University of Minnesota Press. Pp. 209. $5.50. A 

A carefully-done ten year study from the University of Minne- 
sota Counseling Center on closely matched students yielded no re- 
sults when the students were evaluated on anxiety, defensiveness, 
and problem-solving abilities before and after counseling., The de- 

Sign was to process one student (experimental subject) immedi- 

ately through brief counseling while delaying his matched pair 

(control subject), and then giving tests at the start and end of the — 

Counseling for the experimental subject (and hence before the counsa, 

seling of the contro] subject). The design was clever in avoiding the 


, your inflamed tooth will begin to feel better,” 
The authors do a good job in reviewing the field and in setting 
up the design; they make up for this by a rather poor job when the 
egg is on their faces at the end. It is very possible that the students a 


This book inevitably suffers by com 3 : s i 
Results Li „comparison with Campbell’s The 
which a 2 Twenty Five Years Later (Saunders, 1965), 
deft experiment Ports results from the very same source. In a more 
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in experiments reported, it looks a little bit as if too many cooks 
spoiled the broth. 

Jonn C. GOWAN 

San Fernando Valley State College 


Vocational Guidance Readings by George D. Demos and Bruce 
Grant (Editors). Springfield, Illinois: Charles C. Thomas, 1965, 
Pp. xxiv + 521. 

The purpose of this book is to present in an integrated manner 
the multiple aspects of vocational guidance. The articles were se- 
lected as to their appropriateness to the theme of a chapter. No at- 
tempt was made to make a given chapter a comprehensive review 
of a topic. Each selection represents a sample of the typical con- 
cerns in a given area. The selections for the book were obtained 
from the following sources: (a) Vocational Guidance Quarterly 
(24); (b) Personnel and Guidance Journal (15); (c) pamphlets, 
monographs, government publications (11); (d) popular magazines 
(8); (e) newspaper articles (4); (f) original paper (4); and (g) 
book (1). 

The book is divided into the following chapters: (1) Socio- 
economic implications for vocational guidance; (2) Vocational 
development and vocational choice; (3) Industrial, company, and 
vocational careers; (4) Occupational aspirations; (5) Occupa- 
tional values; (6) Prestige factors; (7) Work opportunities; (8) 
Occupational information; (9) Sources of industrial, company, and 
vocational information; (10) Topics to investigate in studying in- 
dustries, companies, and vocations; (11) Classifying and filing 
career information; (12) Evaluating career information; (13) Ed- 
ucational preparation in relation to vocational guidance; (14) 
Teaching of occupations; and (15) Vocational Guidance, Each 
chapter has a short introduction which sets the theme for it. In ad- 
dition each chapter is concluded with a list of suggested additional 
readings. 

The articles represent a mixture of theory and practice. The basic 
premise underlying this approach is the idea that given a the- 
oretical orientation and an example, the reader could put into 
practice in a school setting a procedure that is based on a psy- 
chological premise. In this book of readings, the execution of theory 
into practice is at times blocked by the incompleteness of the 
theoretical presentation, the inappropriateness of the example or the 
lack of one, or by the sequence of articles. For example, there are 
twelve chapters between the discussion of vocational development 
and choice and vocational counseling. There is no mention that 
these ideas are interrelated and that counseling facilitates devel- 
opment and choice. In addition there is an imbalanced presenta- 
tion of the theories of vocational choice: Holland’s theory receives 
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a thorough presentation but other theoretical orientations (e. 
Super, Roe, and Tiedeman) receive little if any attention. 


eral good examples, but the “Introduction” fails to mention some 
of the pertinent factors that should be considered when a filing” 
system is established. À 

There are several areas of vocational guidance which were not 
discussed in this book of readings. Assessment of behavior is one 
of them. Some discussion of test procedures, test interpretations, 
and allied topics should have been made, for they are pertinent to ` 


iG 


ministration of vocational guidance programs which is the concern _ 
of many counselors and administrators is not discussed. If theory 
is to be put into practice, some structure is needed to implement it. 4 
Consequently, a review of organizational and administrative con- _ 
cerns would seem appropriate. 
This book of readings provides only a minimal coverage of the © 
area of vocational guidance. Certain areas like aspirations, values, 
and occupational information have a very good selection of arti- — 
cles in that they give both depth and breadth. In some of the other © 
aspects of vocational guidance, the range of articles is somewhat — 
limited. In one sense this book could be termed a sampler: it stim- | 
eal interest in the field of vocational guidance but fails to satiate 
fi i 
Henry KaczKowsSKI 

University of Illinois 


phe te Making in the Employment Interview by Edward ©. | 
y K Age enon Eagle Publishing Co., Ltd., 1964 Pp. iv + g 
Coe brief volume is largely a compilation of excerpts from doc- 
ral dissertations conducted at McGill University under the di- 
Reda of the author who is Professor and Chairman of the De- ~ 
i ment of Psychology. The volume summarizes nearly a decade 
i aba ty made possible through grants from the Canadian De- 4 
phi anad to at University. Dr. Webster, who is — 
lume, was the principal i i h 
mupported by thes ral principal investigator of the researe 
ieee poate for the extensive research on the merits and limi- 
PENR si t bi pi ace interview reported in the volume are — 
The osaa PS the military and industrial fields of occupation. 
i principal thesis in these experiments is that the employability 4 
th Sais 1s a complex and important matter of concern to both ff 
e employee and employer, and that the successful prediction of 
success in the employment situation is determined by the skill and 
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objectivity with which the interviewer is able to perceive, analyze, 
and integrate the material which is presented to him in the process 
of the employment interview. 

A very complete and revealing review of the literature, ranging 
from the early 1920's to the present day, on the matter of the ac- 
curacy of interview impressions constitutes the major portion of 
Part I on this report. It is based largely on the dissertation work 
of Springbett (1954) and Sydiaha (1958). The major conclusion 
reached by these two authors is that the general consensus of psy- 
chologists concerned with the accuracy of the employment inter- 
view, from E. L. Thorndike to J. P. Guilford, is that the reliability 
of the interview techniques leaves much to be desired. 

The subjects of the McGill experiments were, in the main Per- 
sonnel Officers in the Canadian Army and recruits or men 
awaiting enlistment, The studies reported in this volume, other 
than those by Webster’s students cited above, were conducted by 
Anderson (1961), Crowell (1961), and Rowe (1960). A bibliogra- 
phy of 133 references on the general topic of the employment in- 
terview accompanies the next of this volume. 

The volume itself is organized around the dissertation excerpts 
and unpublished papers on the general issue of decision making in 
the employment interview by the five principal collaborators of 
the text. The reports are interesting, although a critical reviewer 
„can find much fault with the experimental design and conclusions 
of each paper; they tend to be somewhat disjointed and as a whole 
present a rather fragmented picture of the total problem area being 
investigated; and the reader is left at the end with a rather con- 
fused and somewhat pessimistic impression of what was actually 
accomplished by the lengthly project. One of the few positive con- 
clusions drawn from these studies is that the interview as a pre- 
dictor of employability fares much better in presenting negative 
evidence, i.e. identification of the unsuitable candidate. But then, 
this situation is the same for all other psychological measurement 
techniques and devices. Another definite finding was that unfavor- 
able characteristics of the candidate contribute most heavily to 
the decision of rejection for employment. 

The report ends with a list of suggestions offered by the princi- 
pal investigator. These recommendations do not appear to be en- 
tirely forthcoming from the findings reported in the text. Rather 
they seem to go far beyond what was demonstrated by the McGill 
experiments, and as Dr. Webster aptly points out in the introduc- 
tion to Part V “This chapter would have been easier to write had 
the experiments reported not been undertaken.” The reader too is 
left with this last impression. Had the studies not been conducted 
_ and the findings compiled, he might have retained some assurance 

in the value of the interview as an assessment technique. 
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Emotions and the Job by S. G. Roog and C. A. D’Alonzo. Spring- 
field, Illinois. Charles C. Thomas Publishers. Pp. ix -+ 192. 

More man-days of work are lost in industry through the crip- 
pling effect of emotional disturbance and mental health imbalance 
on the part of the worker than is generally realized. Drs. Rogg 
and D’Alonzo document this fact through the scrutiny and ap- 
praisal of the records of more than 100,000 employees of a large 
industrial concern over a period of 35 years. Dr. Rogg is a ‘psy- 
chiatrist, and Dr, D’Alonzo is Assistant Medical Director of the 
E. I. du Pont de Nemours and Company. Both have had wide ex- 
perience in the diagnosis and treatment of emotional problems of 
the American worker. In their present volume they share this in- 
formation with the readers by describing typical examples of thé 
mental health problems of workers, and they attempt to show both | 
the manager and the line worker how to use effectively these data 
and reports. 

To say that the coverage of mental health problems in this vol- 
ume is extensive is certainly an understatement. A total of twenty- 
two topics is covered. They range from a discussion of anxiety and 
fear reactions to the more acute illnesses such as schizophrenia and 
pitti: Also included are brief expositions on motivation, 
aa Aist personality development, and the problems of adoles- 
pire the tobe yah naela accident, proneness, suicidal tendencies, 
the'vemnan tig tobe alah employees constitute a major portion of 
kena e pen ste hi ough the coverage is extensive, it is by no 
meai ve in this brief text. Most of the topics are discussed 
in single chapters, some of which are only two to five pages long. 
The reader, therefore, must be careful not to assume that he can 
pes much sophistication or knowledge of the many concepts to 
which he is superficially introduced by the authors. However, to 
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their credit, the reader is cautioned early by them not to treat the 
contents of the text as recipes for solving human emotional prob- 
lems. 

Many interesting and timely data are presented in the tables 
(labelled figures by the authors) of the latter chapters of the text. 
Particularly relevant and up-to-date are such data as per cent of 
employees taking psychotropic drugs by age and sex; consumption 
of alcoholic beverages before dinner, by age; leading causes of 
death by age groups; and average annual suicide rates. 

The treatment of psychological concepts and principles is quite 
well done for a layman’s consumption. It is accurate, precise, and 
interesting. Perhaps the only lapse in accurate reporting is found in 
the authors’ reference to “Dick” Armstrong rather than Jack Arm- 
strong, the All American Boy, when they are describing the stereo- 
type ideal self-concept. 

Managers and employees both are likely to gain some helpful 
insights into the understanding and solutions of mental health 
problems through the reading of this informative report. 

Prrer F. MERENDA 
University of Rhode Island 


Thinking: From Association to Gestalt by Jean Matter Mandler 
and George Mandler (Editors). New York: John Wiley & Sons, 
1964. Pp. x + 300. $4.95 

This is an excellent compendium of thinking on thought which 
could be used to advantage by any psychologist interested in cogni- 
tion. The volume, which traces a path from Aristotle to Duncker, 
includes previously untranslated work. A scholarly and informa- 
tive running commentary by the editors transforms the volume 
from a collection of readings to one that might well be used as a 
textbook in a course on thinking, accompanied by, perhaps, one of 
the current collections of readings in the cognitive processes, or by 
Computers and Thought, edited by E. A. Feigenbaum and J. Feld- 
man, a collection of research in computer simulation. The latter 
work builds on this foundation to an extent not ordinarily realized, 
as the Mandlers indicate. 

The table of contents provides an outline to the potential user. 
The volume starts with early associationism; Aristotle, Hobb, 
Locke, Hume, Hartley, James Mill, and Bain are sampled. Next, 
thinking and the imageless thought controversy are covered in the 
work of Mayer and Orth, Marbe, Messer, Ach, and Titchener. Last, 
directed thinking and the unit of thought are considered as probed 
by Watt, Ach, Külpe, Müller, Selz, Koffka, Wertheimer, and 
Duncker. 
` The book is handsome and well made. Its price is perhaps a little 
high ($4.95) when one considers the volume’s size. For those will- 
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ing to give thought a place in modern psychology, this book is- 

highly recommended. M4 
EDWARD J. O'CONNELL, JR. 
Syracuse University 


A Primer of Experimental Psychology by Joseph Lyons. New York: 
Harper & Row, 1965. Pp. 322 + xii. $3.50, paperback. 

Although psychologists have always been an argumentative lot, 
the contemporary scene in psychology is one that particularly 
abounds with controversy. The present generation of psychologists 
—ahbove all else—are schooled in critical, logical thinking; to them ` 
most of the old answers are unsatisfactory and everything is to be _ 
questioned. In such an atmosphere it is not surprising to find books = 
being published on controversies within psychology—controversies — 
in learning theory, in perception, and in mental disorders. Another | 
earmark of the current scene is the increasing number of scholarly — i 
paperbacks that are being published. A Primer of Experimental ~ 
Psychology has both of the characteristics referred to; it deals with — 
controversial topics in psychology, and it is a paperback. y 

The author, Joseph Lyons, is a research psychologist at the VA 
Hospital in Lexington and a lecturer at the University of Ken- | 
tucky. Among his research interests are social perception and bodily — 
orientation in space, and he is the author of Psychology and the 
Mi easure of Man as well as the present book. Professor Lyons has a 
flowing, easy-to-read style, but it is obvious that his “experimental 
psychology” is not quite the same as the traditional experimental 
psychology of Woodworth and Schlosberg or Underwood. The ex- 
periments which Lyons cites, with only a few exceptions, have 
more to do with clinical and social psychology than with the “pure” 
experimental psychology of sensation, perception, learning, and _ 
memory. In addition, this book is no primer. Before discussing the 
results of selected investigations, Lyons presents certain back- 
ie opr essential for an understanding of the investiga- 
Biot co a Average beginning, unsophisticated student of psy- 
oa sa still have difficulty comprehending some of the ideas 

oe guments. Consequently, the book will be more useful in up- 
per-division and graduate courses than in the elementary courses ~ 
for which the publishers reportedly intend it 

e » one of a series on experimental nned 
by Harpers, consists of eight chapters: (1). yeaa oe a 
mental Inquiry; (2). An Introduction to What Experimenters Do; | 
(3) Specifying the Experimental Problem; (4) The Logic of In- — 
struments; (5) Experimental Control; (6) The Formal Character- ~ 
istics of Experiments; (7) The Problem of Awareness; and (8) 
R in Experimental Psychology. Throughout the eight chap- | 
rs there is an emphasis on methodology, contemporary issues, 1 
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and the philosophy of scientifie psychology. The book is not a col- 
lection of findings or results of experiments. Rather, the author has 
selected examples of investigations concerned with certain con- 
troversial topics and has organized the text around these topics. The 
six headings of Lyons’ “Library of Controversy” sub-sections are: 
Research and Psychotherapy, Sight and Space, Race and Intelli- 
gence, Mazes and Learning Theory, Experiments on ESP, and 
Learning and Awareness. 

In summary, this is a brief, interesting, well-written book on 
the factors which go into designing, conducting, and interpreting 
the results of experiments in psychology. Lyons gives an overall 
view rather than spending much time on particular matters, and the 
examples which he employs as illustrations of specific ideas and 
methods are uniformly captivating ones. The book will be useful as 
supplementary reading for students who have some background in 
psychology and for the professional psychologists, but it is not 
recommended for the tyro. 

Lewis R. AIKEN, JR. 
Guilford College 


Learning and the Educational Process by J. D. Krumboltz (Edi- 
tor). Chicago: Rand McNally and Company, 1965. Pp. xiii + 
277. 

In the summer of 1964 a conference was held at Stanford Uni- 
versity with the aim of stimulating research on educational prob- 
lems. Support for the conference came from the Cooperative Re- 
search Branch of the U.S. Office of Education and the Carnegie 
Corporation. The idea for the summer program was generated by a 
Social Science Research Council Committee which had as its goal 
stimulation of basic research on educational learning. From the 
many who applied only 40 young researchers could be chosen to 
attend. Fortunately for the rest of us who were not there, a written 
record of ten conference presentations has been made available in 
the present book. 

Learning and the Educational Process was compiled with the 
same goal that motivated the conference; that is, to stimulate re- 
search on educational problems. It is intended that the book serve 
as a resource for instructors and their graduate students, for pro- 
fessional educators, and for researchers in fields related to educa- 
tion. 

A review of the general topics covered in the volume reveals three 
areas of emphasis: research methodology, motivation, and lan- 
guage. Three chapters have as their theme the methodology for re- 
search on educational problems. Robert Gagné argues for the state- 
ment of educational objectives in terms of human performance. 
Lawrence M. Stolurow suggests the use of a teaching model for 
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investigations into problems of school learning. His comments 
about the distinctions between descriptive and predictive models 
seem particularly appropriate. In a chapter titled “School Learn- 
ing over the Long Haul” John B. Carroll asks for longitudinal re- 
search into the long-term effects of curricula. He identifies apti- 
tudes, perserverance (motivation), opportunity to learn, the quality 
of instruction, and ability to comprehend instruction (language) 
as relevant variables needing longitudinal study. 

John W. Atkinson, Daniel E. Berlyne, and Jerome Kagan write 
about their particular areas of concern within the general topic of 
motivation. Atkinson presents a clear statement of the effects of 
need for achievement and fear of failure on human performance, 
His work is an example of the methodological approach about 
which Gagné writes. The role of conflict in arousing curiosity drive 
and of various methods of presenting information to reduce this 
drive is discussed by Berlyne. It is worth noting here that at sev- 
eral points the authors of the various articles seem to be in dis- 
agreement. Atkinson argues persuasively against the use of drive 
reduction theories in education; Berlyne uses one. Kagan’s con- 
tribution is concerned with the differences in behavior of children 
identified as impulsive or reflective, 

À Ernst Rothkopf’s article on “, . . Problems in Written Instruc- 
tion” has more general interest than its title might indicate. It 
bridges the gap between the “motivationalists” and those whose 
primary concern is with language. It is, along with Atkinson’s 
chapter, a good methodological model for the educational re- 
searcher. The major focus of his concern is the behavior involved 
in successfully learning from written material, something which 
Rothkopf calls “mathemagenic” behavior. 

i The positive effects of middle class homes and the negative ef- 
ects of lower class homes upon the language development of chil- 


brief report of the findings from his longitudinal study of the lan- 
oe Proficiency of school children. Evan R. Keislar and Larry 


This is a brief overvi 
general competency of 


areas of research which these authors are pursuing. Concern for 


arch into ivati 
represent the major themes of this book. language and motivation 


š commended for choosing the 
articles presented here, However, it must also be said that no at- 
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tempt is made to provide a structure for the overlap and diversity 
in the book. Occasionally, one author referes to the work of another 
within the book, but this seems less integration than afterthought. 
Krumboltz hopes that the dissonance produced by the diversity of 
views “will result in constructive efforts at resolution.” The con- 
structive direction of dissonance resolution could have been made 
more likely, however, by the introduction of a summary chapter, 
or, perhaps, the exchange of comments by the authors on one 
another’s papers. 
Learning and the Educational Process or, what would have been 

a more appropriate title, “Research and the Educational Process,” 
hits the mark. It is provocative reading for those who identify, or 
would like to identify, educational research as their working do- 
main, 

Loren S. BARRITT 

The University of Michigan 


School Psychology by Roger Reger. Springfield, Illinois: Charles C. 
Thomas, 1965. Pp. xv + 213. $7.75. 

Within the last two decades the school psychologist has emerged 
as a significant figure in the educational system. Unfortunately, the 
profession, in its evolution, has never had a well defined role 
within the educational system. Because of this, educators fre- 
quently question the value of the school psychologist’s services and 
relegate the psychologist to the role of test administrator and 
reporter. 

Roger’s book represents a recent trend on the part of several 
concerned individuals to clarify the aims and goals of the profes- 
sional school psychologist. The book is the work of a practicing 
school psychologist who has been taking some very critical looks 
at the practices in the profession. This worthwhile, though diffi- 
cult task is accomplished by discussing a number of contemporary 
problems, practices, and issues in the field which have hindered the 
development of the profession. 

The book is divided into three main parts: (a) the roles and 
identity of the school psychologist, (b) issues in school psychology, 
and (c) a theory of special education. In part one Reger sets forth 
his major argument that the school psychologist should be con- 
sidered an educator, by first pointing to the present inadequate role 
of the psychologist and then by elaborating on the new roles and 
identity the profession should seek. He contends that the contempo- 
rary emphasis of the profession on pathology, its identification with 
clinical psychology, and particularly its psychometric orientation 
have lessened the effectiveness of the psychologist and his contribu- 
tion to the education of the child. 

“Educational philosophies and practices continue to change, but 
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school psychologists have put themselves in the position of being 
followers, rather than leaders, of the processes of change. Psy- 
chologists, trained in theory, practice, and research, probably will 
continue for some time to come to administer their tests, score 
these tests, and smugly interpret the results to receptive school per- 
sonnel. And while psychologists continue to talk about score pat- 
terns, ego boundaries, and superego conflicts, ‘real IQs’, ‘poten- 
tials’, ‘prevention’, and a salad of other meaningless nonsense, 
change will take place all around.” 

Reger believes that a more meaningful identity for the school 
psychologist would be in a model which resembles the college aca- 
demician, enabling the psychologist to function as an educator 
rather than a clinician. The primary function of the psychologist 
within the role of educator would be one of planning educational 
programs for children. With the psychologist placed high in the 
administrative structure, he would have the flexibility to function, 
not in the narrowly prescribed role of psychotherapist or test ad- 
ministrator, but as a teacher, programer, and researcher. The psy- 
chologist’s basic purpose in the public school system would be to 
improve the quality and effectiveness of the educational process 
through the use of psychological knowledge. In essence Reger con- 
tends that the school psychologist should be viewed in the role of 
an educational practitioner as well as a scientist utilizing the latest 
methodology and techniques in the design of educational plans. 

_Although not all readers of this journal would agree with Reger’s 
view of psychological tests as being “primitive instruments” which 
secretaries can administer and interpret, one is impressed with his 
attempt to conceive of the psychologist, not as a technician, but 
rather as a professional expert with one of many specialties being 
in the area of psychometrics, 

Part two, which constitutes almost half of the book, is devoted to 
contemporary issues within the profession, including a discussion of 
(a) the predictive validity of 1Q tests, (b) the lack of much needed 
Tesearch by school psychologists in their profession, (c) the value 
of psychotherapy in general and the question of the necessity for 


h A openness of information” to par- 
ents in psychological reports, and (£) the interaction between the 
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clarification may tend to confuse and prejudice the lay reader, 
particularly since Reger also points to the necessity for psycholo- 
gists to be well trained in psychological testing. 

Part three of the book outlines an educational theory for ex- 
ceptional children and stresses the need for a theory which consid- 
ers the needs of the individual child rather than builds programs 
around “classifications” of children presumed to have the same 
characteristics. The ideas presented here are worthy of considera- 
tion, but unfortunately the inclusion of this section does not add to 
the author’s main thesis, namely defining the goals and functions of 
the school psychologist. The impression one receives is that the 
author has much to say about school psychology, educational phi- 
losophy, and professional competency, and for the sake of ex- 
pediency has placed them all in one book. 

This book reflects more the author’s own experiences and views 
than the documented opinions of others. As such, the references are 
not integrated very well into the chapter discussions, but they are 
quite substantial and extensive. The chapters within each section 
are organized well, but at times they tend to be redundant. For the 
most part, the language which is nontechnical should not pose dif- 
ficulties for lay people. 

Throughout the book, Reger has taken a very critical view of the 
entire profession of school psychology, and a number of psycholo- 
gists reading it will no doubt disagree and even be irked at his 
analysis of many of the present practices in the field and by his 
suggested role for the school psychologist. It is a book, however, 
that should have been written years ago. The author has made a 
sincere and worthwhile effort to clarify the structure and the goals 
of the professional school psychologist. As such, Reger has made a 
significant contribution. Although this volume cannot be considered 
a textbook in the usual sense, educators, practicing psychologists, 
and graduate students would find it worthwhile to read and con- 
sider carefully. 

Ratpu B. VACCHIANO 
Fairleigh Dickinson University 


Psychology and Education of the Gifted: Selected Readings by 
Walter B. Barbe (Editor). New York: Appleton-Century- 
Crofts, 1965. Pp. x + 534. 

This book of readings on the psychology and education of the 
gifted consists of 54 papers, primarily journal articles, dealing with 
philosophical presentations, discussions of different points of view, 
reviews of literature on specific topics, and significant research re- 
ports. Most of the papers were published before the sixties; the 
editor points out that “inclusion was determined by importance in 
the field rather than recency of publication.” 
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The editor has attempted to bring together a collection of papers 
which present significant, and sometimes contradictory, points of 
view; he has done remarkably well in collecting much of the out- 


{ Part T, Measurement of Giftedness, includes three articles on 
identification and testing the gifted ; three articles were selected on 
non-intellectual factors and giftedness, 


been influential throughout this cen 
he reviewer hag One reservation about this book regarding the 
n the measurement and identi- 


is that it provides important Samples of the thinking which has 


BOOK REVIEWS 1113 


fication of giftedness. Inasmuch as the assessment of giftedness 
represents a major area of concern to both psychologists and edu- 
' cators alike, a greater emphasis should be devoted particularly to 
the measurement of higher mental processes of the gifted student. 
Bert W. WESTBROOK 
North Carolina State University 


Teaching the Culturally Disadvantaged Pupil by J. M. Beck and 
R. W. Saxe (Editors). Springfield, Ohio: Charles C. Thomas, 
1965. Pp. xvi + 335. 

Less than a decade ago the rising tide of delinquency urged upon 
the government the need to promote research of and training for 
unemployed school dropouts. Dropouts, it was believed, provided 
the soil upon which delinquency breeds. In addition to stop-gap 
measures, the Kennedy era ushered in developmental programs for 
potential dropouts. The first programs were begun in economically 
deprived communities and neighborhoods, where ignorance, un- 
employment, and inertia were believed to flourish. The jeremiad 
Michael Harrington bruised our complacency and made us stand 
up and take notice of the mores, hopes, and disenfranchisement of 
a neglected fifth of the population. 

Available now are the records of university based endeavors as 
well as of public-education based undertakings such as those in 
Philadelphia, New York, St. Louis, and San Francisco. The data 
do appear to be helter-skelter. In all fairness to the pioneer work- 
ers, with little theory as a guide, a purely empirical approach was 
justified and salutory. The time has come, though, to pause and re- 
view, synthesize, and select the most feasible approaches to edu- 
cating those from humbler circumstances. Schools of education are 
preparing to increase their efforts in developing a pedagogy for the 
poor. The outstanding school administrators have insisted that 
their teachers take in-service courses on the disadvantaged. The 
book by Beck and Saxe meets a demanding market. 

Will the book satisfy this market? Today’s knowledge about 
“the disadvantaged” may be likened to human prenatal develop- 
ment. No longer a zygote, the organism is maturing into a fetus. 
The educational mother is just beginning to suspect that she is 
pregnant. The Beck and Saxe exorcising of the fetus at this time 
reveals in four chapters what are the basic characteristics of the 
disadvantaged, in eight chapters what the elementary school could 
do for him, and in three chapters what the community could do for 
him. The impression this reviewer has is this: The differences be- 
tween the fetus under discussion and that of any other species is 
hardly recognizable. Several of the 17 contributors to the book ac- 
knowledge and are even apologetic about the modicum of viable 
knowledge they impart. Even as they awaited their galley proofs, 
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cell division was constantly taking place. Information was taking 
on new structures. While the stethoscope the editors applied to 
the disadvantaged might have detected the heartbeat of the fetus, 
it may only have been the mother’s heartburn, 

The spirit which guided this book is to 


mental deprivation,” and to remove more than half of the academic 


Posited a “social system” input and behavior output model. The so- 
cial system bifurcates into nomothetic and idiographic lines, This 


Misi ated Eom others. | ong the disadvantaged, the atti- 
with torn ce re ie a Negative, Their children play infrequently 
iow ct ildren. Reading Matter is absent from their 
es. Among the behavioral attributes identified by the writers 
hing were inferior auditory discrimination, poor learn- 

One may ask what kind of 


p 
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ure it, and to the formulation of an extensional definition of it that 
would be useful to any external observer. If the functional rela- 
tionship between change in such attributes as reading ability with 
remedial treatment is no different for a group “labeled” disad- 
vantaged than it is for those without the label but with the same 
ability and with the same treatment, there is little reason to re- 
tain the label. 

The chapters dealing with teaching of arithmetic, language, so- 
cial studies, art, and music are particularly valuable to a teacher. 
The games, exercises, and projects emphasize body involvement. The 
projects range from writing Haiku to pen pal clubs for potential 
dropouts. Many of these exercises are recommended for those stu- 
dents who do not bare the badge of “the disadvantaged.” From the 
tone of these chapters this reviewer felt that the authors were de- 
scribing youth who were at a disadvantage in school in reading, 
in arithmetic, or in science. Whether the disadvantage is due to 
cultural forces, due to the range of stimuli which a family permits 
its children to experience, or due to a set of school standards alien 
to some youth is debatable. 

Despite its limitations, the book does contain valuable messages 
to the educator. From the book one may see how vital is concept 
learning, how important it is to teach the child to learn how to 
learn, and what is the motivational value of giving a child some- 
thing that is his own. Some other salient points were to acquaint 
present oriented youngsters with the past and to learn the world 
from the point of view of the child. 

Redundancy abounds in the book. The works of Bernstein on 
public and private language, of Deutsch on concept development, 
and of Clark on self-fulfilling prophesy are cited in different chap- 
ters to fix the same points. In the chapter on language arts this re- 
viewer hoped to see the pedogogical implications of Bernstein’s 
works. This was one of the chapters, however, in which his work 
was not cited. In conclusion, thinking about the disadvantaged is 
still inchoate. Many of the novel instructional aids described merit 
classroom trials. In addition to teaching youth at a disadvantage in 
school, it would be well to learn from them during this next five year 
period. Hopefully, the editors will examine the embryo. 

Norman M. Cuansxy 
North Carolina State University 


Reading Comprehension for Scientists by Richard H. Bloomer. 
Springfield, Illinois: Charles C. Thomas, 1963. Pp. xiii +213. 
$8.75. 

The need for materials and practice exercises which will help per- 
sons to improve comprehension of scientific materials is apparent. 

If it could be demonstrated that such materials were effective, they 
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would probably be widely used. It is on this point that Reading 
Comprehension for Scientists must be questioned. 

This book of practice exercises is based on passages taken from 
science texts from a broad range of subject matter and arranged 
in “levels” of difficulty. Every tenth word is deleted in each par- 
agraph, and these deleted words are dropped to the end of the para- 
graph in scrambled order. The Key in the back of the book gives 
the correct order of these words and a “critical score.” If the user 
does not reach this critical score, he will do the next passage of par- 
allel difficulty; if he reaches this score he will move up to the first 
exercise in the next level. 

The author indicates in the index that this approach to practice 
will teach a student to attend to what he is reading, and teach him 
to postpone his conclusions until he has read the total context, and 
thus he will learn the “elemental scientific concepts and the sci- 
entific language pattern” (p. vii). 

The materials have face validity. Coming from a wide range of 
textbooks, they are concerned with a large number of scientific 
topics. However, further questions of validity must be raised. 

All exercises are based on the cloze technique. The author does 
not state how the choice for deletions was made. There are two 
usual ways; the so-called structural and the lexical. There has been 
Some research to indicate that cloze tests based on these two tech- 
niques do not measure the same thing. But a more basic question 
here is whether the cloze technique used in exercises to teach com- 
prehension skills are valid at all. It seems that the question cannot 
be answered directly. The author presents no evidence by which the 
use of this book of exercises can be evaluated and the literature 
contains a paucity of such evidence. One must, therefore, turn to 
evidence which has been Presented to validate cloze tests, 


Certainly after trying the exercises in this book and i 
j watching 
— adults try them and either becoming discouraged and quit- 
g or approaching them as one would with a puzzle, the reviewer 
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Therefore, it would seem that the following questions need to 
be answered before this book is used as the basis of a program pur- 
porting to develop reading skills necessary for comprehending sci- 
entific material. 

1. What skills are necessary for successful reading of scientific 
material? (Theoretically they differ only in emphasis from those 
necessary for reading other types of material.) 

2. Does the cloze technique applied to scientific material develop 
these skills (after they are defined)? 

3. Will persons in need of developing skills to improve the 
reading of scientific material apply themselves to this set of ex- 
ercises long enough to learn these skills? 

One other concern. The retail price of this book is $8.75. Stu- 
dents need not put their answers in the text, but an answer booklet 
should be made available if the book is to be reused. A soft cover 
would reduce the cost of the text somewhat. Surely potential users 
will consider cost when sampling texts to recommend to students. 

Frances Tricas, Chairman 
The Committee on Diagnostic 
Reading Tests, Inc. 

Mountain Home, N. C. 
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A Guide to Evaluating Self-Instructional Programs by Paul I. 

Jacobs, Milton H, Maier, and Lawrence M. Stolurow. New 

York: Holt, Rinehart and Winston, Inc., 1966. Pp. ix + 84. 

The title of this book is somewhat a misnomer. It might more 
appropriately be called Understanding, Selecting, and Evaluating 
Self-Instructional Programs. The first third of the book is an ex- 
planation of what programmed instruction actually is, the second 
third is devoted to how to select a program, and the last third to 
evaluation. It must be emphasized that as stated in the preface, 
this book “may profitably be read by teachers, principals, curricu- 
lum specialists and other school administrators who may be un- 
acquainted with this area.” The writers might have also added 
(but did not) “and not so profitably read by psychologists, educa- 
tors, or any other scholars.” This book is not recommended for the 
typical reader of EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT; 
he would find the first two-thirds of the book exceedingly fundamen- 
tal and the latter third mildly interesting. As an example, the table 
of contents lists two-thirds as many titles as there are pages in the 
book with such titles as “What Do Programs Look Like,” “On What. 
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Educational Principles is Programmed Instruction Based,” or 
“What Types of Response Do Programs Require the Student To 
Make,” In a sense, this book is programmed, but there are not di- 
rections in the front for certain people to “skip to page 43”! This 
writer is convinced, however, that the text is certainly a guide for 
those not in the educational and psychological measurement, area. 
It could prove very handy for a teacher of Spanish who wished to 
understand, select, and somewhat superficially evaluate a program 
for her area. 

The volume is almost totally functional with little time spent on 
the theoretical or empirical research aspects. For example, the 
difference between branch and linear programming is explained al- 
most entirely mechanically with no tie-in to theoretical concepts. 
Names of famous people working in the area, references to re- 
search conducted, films available, names of sources of instructional 
programs are sprinkled throughout the book. In view of this, it is 
obvious that the use of the word “Guide” in the title is an ap- 
propriate word. It is somewhat lamentable, however, that a greater 
attempt to insert some thought-provoking material was not made. 
The latter third of the book does provide some basic information 
on sampling problems, the need for some basic statistics, and the 
like. Toward the end of the third part, an actual evaluation study 
done with the Denver Public Schools is presented. If, however, a 
piece of research was contemplated by a reader involving the usual 
statistics, further help would be needed. 

In conclusion, this short contribution is definitely recommended 
as an excellent source of fundamental information on understand- 
ing, selecting, and evaluating programmed instruction materials, 
particularly for the uninformed. 

Purr S. Very 
University of Rhode Island 


The Hidden Dimension by Edward T. Hall. New York: Double- 
day and Company, Inc., 1966. Pp. xii -+ 193. 

_Hall’s very interesting and readable work is a functional com- 
bination of the Sapir-Warf hypothesis and social pathology. Begin- 
ning with the research work by C. R. Carpenter, John B. Calhoun, 
and others, Hall extends into the realm of human metropoli- 
tan life this basic finding: the crowding of rats in a limited amount 
of space yields abnormal patterns of behavior that can lead to pos- 
sible extinction of the entire population. 

Obviously, it takes a considerable degree of generalization to go 
from these „experimental laboratory conditions to discussions of 
inter- and intra-cultural relationships. Hall does this extrapola- 
tion on the basis, primarily, of personal experience, with some sup- 
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port from sociological studies. He is presenting, in effect, an hy- 
pothesis: ` 

“The implosion of the world population into cities everywhere 
is creating a series of destructive behaviorial sinks more lethal than 
the hydrogen bomb.” 

This sink is caused by variables ignored by government and 
society; i.e. the number of square feet per person and limitation im- 
posed on his perceptual space. When limited, man is uncomfortable 
and antagonistic. The variables which help to control man’s emo- 
tions include his visual, auditory, kinesthetic, olfactory, and ther- 
mal perceptions of his geographical life space. These factors, which 
are culturally learned, differ from culture to culture—and perhaps 
more importantly in the United States, from sub-culture to sub- 
culture. 

If, as Hall states, the population density in already overcrowded 
cities is to be ameliorated, architects must consider these cultural 
values in building life spaces more conducive to ecological har- 
monious relationships. This is not now being accomplished, and, 
the social upheaval being felt in Los Angeles, Chicago, Cleveland, 
and other cities may be, in part, due to the ecological disparity 
between man’s cultural heritage and his physical environment. 

“It is a mistake of the greatest magnitude to act as though man 
were one thing and his house or his cities, his technology, or his 
language were something else.” Hall has written this book, it seems, 
to call for operational research activity to study the elements in- 
volved in man’s life space. Quantitative measures of such variables 
are not in evidence, and the current attempts at solutions through 
Headstart or Wingspread Programs tend to deal with the result, not 
the cause. As potentially valuable or helpful as such programs 
might be, it is essential that long-range programs and machina- 
tions be developed which can study, measure, and develop large 
scale city plans accounting for “The Hidden Dimension.” 

This reviewer found this book upsetting. It explores areas of 
human behavior which tend to be ignored. If, however, any sig- 
nificant dent on the “culturally deprived” is to be made, or if peo- 
ples from different environments and cultures are to be welded suc- 
cessfully (nonantagonistically), this book cannot be ignored. It 
should be required reading for all educators operating in the field of 
the culturally disadvantaged, or the culturally different. 


RICHARD E. SPENCER 
University of Illinois 
Champaign 


